CS 6140: Machine Learning
GENERAL INFORMATION 
Instructor: Prof. Ehsan Elhamifar
Instructor Office Hours: Mondays, 4:30pm5:30pm, 310E WVH
Class: Mondays and Wednesday 14:50—16:30, Ell Hall 312
TA1: Ankur Bambharoliya (bambharoliya.a [at] husky.neu.edu), Office Hours: Fridays, 3pm4pm, 462 WVH
TA2: Yash Bhavsar (bhavsar.y [at] husky.neu.edu), Office Hours: Mondays, 11am12pm, 462 WVH
Discussions, Lectures, Homeworks on Piazza

NOTICE 
The class starts from January 14.

DESCRIPTION 
This course covers practical algorithms and the theory for machine learning from a variety of perspectives. Topics include supervised learning (generative/discriminative learning, parametric/nonparametric learning, deep neural networks, support vector machines) and unsupervised learning (clustering, dimensionality reduction). The course will also discuss recent applications of machine learning in computer vision, data mining, natural language processing and robotics.

PREREQUISITES 
Introduction to Probability and Statistics, Linear Algebra, Algorithms.

SYLLABUS 
Supervised Learning
Linear regression, overfitting, regularization, sparsity
Logistic regression
Naive Bayes
Neural networks and deep learning: DNNs, CNNs, RNNs
SVM, Perceptron and kernels
Decision trees and instancebased learning
Unsupervised Learning
Clustering: kmeans, spectral clustering
Dimensionality reduction: PCA, Kernel PCA, Autoencoders
Expectation Maximization

GRADING 
Homeworks are due at the beginning of the class on the specified dates. No late homeworks or projects will be accepted.
Homeworks: 4 HWs (40%)
Project (30%)
Final Exam (30%)
Homework consist of both analytical questions and programming assignments. Collaboration on HWs is not allowed, unless specified. Programming assignments must be done via Python. Both codes and results of running codes on data must be submitted.
Exams consist of analytical questions from topics covered in the class. Students are allowed to bring a single cheat sheet to the exam.

TEXTBOOKS 
[CB] Christopher Bishop, Pattern recognition and machine learning. [Required]
[KM] Kevin P. Murphy, Machine Learning: A Probabilistic Perspective. [Optional]
[KF] Daphne Koller and Nir Friedman, Probabilistic Graphical Models. [Optional]

READINGS 
Lecture 1: Introduction to ML, Linear Algebra Review
Lecture 2: Introduction to Regression, Convex Functions and Optimality
Lecture 3: Linear Regression: Closedform Solution, Gradient Descent and SGD, Basis Function Expansion
Lecture 4: Robust Regression, Overfitting, Regularization
Lecture 5: Hyperparameter Tuning, Cross Validation, Probability Review
Lecture 6: Maximum Likelihood Estimation, Maximum A Posteriori (MAP) Estimation
 Chapter 2 and 3 from CB book.
Lecture 7: Classification, Logistic Regression, Parameter Learning via Maximum Likelihood
 Chapter 4.3 from CB book.
Lecture 8: Softmax Regression, Overfitting, Discriminate vs Generative Modeling, Generative Classification
 Chapter 4.2 from CB book.
Lecture 9: Generative Classification, Naive Bayes
 Chapter 4.2 from CB book.
Lecture 10: Generative Classification, Naive Bayes
 Chapter 4.2 from CB book.
Lecture 11: Convex Optimization, Lagrangian Function, KKT Conditions
 See lecture notes on piazza.
Lecture 12: Suport Vector Machines: Vanilla SVM, Dual SVM
Lecture 13: Suport Vector Machines: SoftMargin SVM, Kernel SVM, MultiClass SVM
Lecture 14: Dimensionality Reduction: Principal Component Analysis
Lecture 15: Neural Networks
Lecture 16: Neural Networks: Training, Forward and Back Propagation

ADDITIONAL RESOURCES 
Probability Review
Linear Algebra Review

ETHICS 
All students in the course are subject to the Northeastern University's Academic Integrity Policy. Any submitted report/homework/project by a student in this course for academic credit should be the student's own work. Collaborations are only allowed if explicitly permitted. Per CCIS policy, violations of the rules, including cheating, fabrication and plagiarism, will be reported to the Office of Student Conduct and Conflict Resolution (OSCCR). This may result in deferred suspension, suspension, or expulsion from the university.

