Home     MCADS Lab     Research     Publications     Activities     Codes     Teaching


EE 290: Advanced Topics in Computer-Aided Design: Modeling and Analysis of High-Dimensional Data


DESCRIPTION

The increasing amounts of high-dimensional data across different science and engineering disciplines requires efficient algorithms for analyzing and uncovering the structure in the data. This course covers state-of-the-art algorithms for modeling and analysis of high-dimensional datasets with low-dimensional structures. The course introduces different classes of methods, including latent variable models, spectral clustering-based methods, and sparse/low-rank representation-based algorithms, and covers the theoretical tools for the analysis of each class of methods. In addition, the course provides students with implementation experience of different algorithms and discusses applications of these tools in signal/image processing, controls, computer vision and bio-informatics.

PREREQUISITES

Linear Algebra, Introduction to Probability. Additional background from convex analysis, optimization, ect., will be introduced in the course as necessary.

GRADING

There will be 4 homeworks, which include both analytical exercises and programming assignments in MATLAB or Python. The course will have a final project (4th HW), which will be on analytical problems or research-related applications. The final project can be done individually or in teams of two students.

OFFICE HOURS

Fridays, 4:00pm--5:00pm, 337 Cory Hall.

SYLLABUS
  1. Modeling and Analysis of Data in a Single Subspace

    • Principal Component Analysis (PCA), Factor Analysis, Probabilistic PCA

    • Robust Principal Component Analysis: Dealing with Missing Entries and Errors

    • Kernel Principal Component Analysis: Dealing with Data in Nonlinear Manifolds

    • Other Extensions of PCA

  2. Modeling and Analysis of Data in Multiple Subspaces

    • Iterative Algorithms: K-Subspaces, Mixture of Factor Analyzers

    • Algebraic Algorithms: Generalized PCA

    • Spectral Clustering-Based Algorithms: Local Subspace Affinity, Spectral Curvature Clustering, Sparse Subspace Clustering

  3. Sparse and Low-Rank Recovery Theory

    • Sparsity and Properties of Different Norms

    • Coherence, Null-Space Property and Generalization to Subspaces

    • Restricted Isometry Property (RIP) and Rank-RIP

    • Group-Sparsity, Multiple Measurement Vector Problems

  4. Sparse and Low-Rank Optimization Methods

    • Convex Algorithms: Proximal Methods, Alternating Direction Methods, Non-smooth Convex Optimization

    • Greedy Algorithms: Matching Pursuit, Orthogonal Matching Pursuit, Subspace Pursuit

    • Homotopy Algorithms

  5. Applications

    • Compressive Sensing

    • Dictionary Learning

    • Compression and Clustering of Visual Data

    • System Identification

    • Other Application Areas: Energy Systems, Bio-informatics

READINGS
HOMEWORKS

TEXTBOOKS

  • [ME] M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing, Springer.

  • [KM] K. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press.