EECS 453: Principles of Machine Learning

Instructor: Prof. Laura Balzano, Prof. Qing Qu, Prof. Lei Ying

EECS 351, or EECS 301, or any linear algebra courses

The class will cover basic principles in machine learning, such as unsupervised learning (e.g., clustering, mixture models, dimension reduction), supervised learning (e.g., regression, classification, neural networks & deep learning), and reinforcement learning. For each topic, key algorithmic ideas/intuitions and basic theoretical insights will be highlighted.

This is an entry-level machine learning course targeted for senior undergraduate EE and CE students, and junior master students outside the area of Signal and Image Processing & Machine Learning. Compared to EECS 445, this course places slightly greater emphasis on mathematical principles and is better suited for students who have limited experience with programming and machine learning.

Course Materials: slides and videos will be accessed via Canvas (TBA). Tentative topics that will be covered in this course are supervised learning, unsupervised learning, and reinforcement learning:

  • Basics of probability, linear algebra, and optimization
  • Regression and linear prediction
  • Support vector machines and kernel methods
  • Deep neural networks
  • Dimension reduction: PCA, autoencoder
  • Clustering (Kmeans, Mixture of Gaussians, EM)
  • Representation learning: nonnegative matrix factorization, dictionary learning

Assessment: (i) 5 homework assignments (40%), (ii) mid-term exam (30%), (iii) course projects (25%), (iv) participation & course evaluation (5%)

Textbook: We recommend the following books and articles, although we will not follow them closely.

Related courses:

  • EECS 445. Introduction to Machine Learning
  • EECS 453. Applied Matrix Algorithms for Signal Processing, Data Analysis, and Machine Learning
  • EECS 505. Computational Data Science and Machine Learning
  • EECS 545. Machine Learning

Course Syllabus (Note: the schedule is tentative, and is subject to change during the semester.)

WeekDate TopicContentsHomework, Review
Week-1-108/29Introduction (Remote)Course overview 
Week-1-208/31Supervised Learning (Remote)Introduction to supervised learning, linear models, regularizationLinear Algebra Review 
Week-2-109/05Labor DayNo class 
Week-2-209/07Supervised LearningLearning TheoryProbability Review, HW1 Release 
Week-3-109/12Supervised LearningLinear regression I 
Week-3-209/14Supervised LearningLinear regression IIPython Review 
Week-4-109/19Supervised LearningLinear Classifiers 
Week-4-209/21Supervised LearningLinear Discriminant AnalysisHW1 Due, HW2 Release 
Week-5-109/26Supervised Learning (remote)Logistic regression 
Week-5-209/28Supervised Learning (remote)Optimization methods I 
Week-6-110/03Supervised LearningOptimization methods II 
Week-6-210/05Supervised Learning Support vector machine (SVM) IHW2 Due,  HW3 Release  
Week-7-110/10Supervised Learning Support vector machine (SVM) II 
Week-7-210/12Supervised Learning Support vector machine (SVM) III 
Week-8-110/17Fall Study DayNo class 
Week-8-210/19Supervised LearningDual SVM HW3 Due
Week-9-110/24Supervised LearningNonlinear models, kernel methods 
Week-9-210/26Supervised LearningIntroduction to deep neural networks I 
Week-10-110/31Supervised LearningIntroduction to deep neural networks II 
Week-10-211/02Supervised LearningIntroduction to deep neural networks III 
Week-11-211/09Unsupervised LearningIntroduction to unsupervised learning, clustering problem, K-meansProject Proposal Due, HW4 Release
Week-12-111/14Unsupervised LearningK-means, mixtures of Gaussian, expectation maximization 
Week-12-211/16Unsupervised LearningDimension reduction, PCA 
Week-13-111/21Unsupervised LearningDimension reduction II 
Week-13-211/23ThanksgivingNo Class 
Week-14-111/28Unsupervised Learning (Remote)Representation learning, matrix factorizationHW4 Due, HW5 Release   
Week-14-211/30Unsupervised Learning (Remote)Autoencoder & self-supervised learning 
Week-15-1 12/05Unsupervised LearningGenerative ModelsHW5 Due 
Week-15-2 12/07 Final Presentation