Fall 2024, University of Waterloo.
Lecture slides and video lectures (recorded offline) will be linked here. Please note that the topics and schedule for all future lectures are tentative.
The suggested readings make use of the following abbreviations to refer to textbook sources:
UML: Understanding Machine Leaning: From Theory to Algorithms. Shai Shalev-Schwartz and Shai Ben-David.
ESL: Elements of Statistical Learning. Trevor Hastie, Robert Tibshirani, and Jerome Friedman.
ISL: Introduction to Statistical Learning. Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani.
PML: Probabilistic Machine Learning. Kevin P. Murphy
PRML: Pattern Recognition and Machine Learning. Christopher M. Bishop
PML2: Probabilistic Machine Learning: Advanced Topics Kevin P. Murphy
| LECTURE | DATE | TITLE | MATERIALS | SUPPLEMENTARY READINGS |
|---|---|---|---|---|
| 0 | 05/09/2024 | Logistics & Introduction | Slides Video Lecture |
N/A |
| 1 | 10/09/2024 | Halfspaces & The Perceptron Algorithm | Slides Video Lecture Perceptron Video |
UML Section 9.1 ESL Section 4.5 Yaoliang Yu's Lecture Notes Varun Kanade's Lecture Notes |
| 2 | 12/09/2024 | Linear Regression & Loss Function Design | Slides Video Lecture |
UML Section 9.2, 11.2 ESL Section 3.2, 3.4, 7.10 ISL Sections 3.1-3.2, 5.1, 6.2 Gautam Kamath's video on Rewriting Loss Functions |
| 3 | 17/09/2024 | Maximum Likelihood Estimation & Information | Slides Video Lecture |
UML Section 24.1 |
| 4 | 19/09/2024 | Non-Parametric Methods: KDE, k-NN, and K-Means | Slides Video Lecture |
UML Section 19, 22.2 ESL Section 2.3.2, 6.6.1, 13.3, 14.3 ISL 2.2, 12.4.1 |
| 5 | 24/09/2024 | Logistic Regression and Numerical Optimization | Slides Video Lecture |
UML Section 9.3 ESL Section 4.4 ISL 4.3 PML 10-10.2.3 PRML 4.3.2 |
| 6 | 26/09/2024 | Maximum Margin Classifier and Constrained Optimization | Slides Video Lecture |
UML Section 15.1 ESL Section 4.5.2 ISL Section 9.1 PML Section 8.5, Section 17.3-17.3.2 PRML Section 7.1 (not including 7.1.1) |
| 7 | 01/10/2024 | Soft Margin Classifier | Slides Video Lecture |
UML Section 15.2,15.5 ESL Section 12.1-12.3.2 ISL Section 9.2-9.3 PML Section 17.3.3-17.3.4 PRML Section 7.1.1 |
| 8 | 03/10/2024 | Kernel Methods | Slides Video Lecture |
PRML Section 6.1-6.3, 3-3.1.0 UML Section 16 ESL Section 5.8 PML Section 17.1 ISL Section 7-7.3 |
| 9a | 08/10/2024 | Bayesian Inference | Slides Video Lecture |
PRML Section 3.1,3.3 |
| 9b | 10/10/2024 | Gaussian Processes | Slides Video Lecture Sampling Animation |
PRML Section 6.4 PML Section 17.2 |
| 10 | 10/10/2024 | Decision Trees | Slides Video Lecture |
ESL Section 9.2 ISL Section 8.1 UML Section 10 |
| 11 | 10/22/2024 | Ensemble Methods | Slides Video Lecture |
PRML Section 3.2, 14 PML Section 18 ESL Section 7.1-7.3, 8.2, 8.7, 10.2, 10.10 ISL Section 8.1-8.2 |
| 12 | 10/24/2024 | Expectation Maximization & Gaussian Mixture Models | Slides Video Lecture |
PML Section 8.7 ESL Section 8.5 PRML Section 9.1-9.5 UML Section 24.4 |
| 13 | 10/31/2024 | Multi-Layer Perceptrons and Deep Learning | Slides Video Lecture |
D2L Section 5 DL Section 6,7 ISL Section 10-10.2, 10.6 |
| 14 | 05/11/2024 | Convolutional Neural Networks | Slides Video Lecture |
D2L Section 7 DL Section 9 ISL Section 10.3 Fukushima, 1980 |
| 15 | 07/11/2024 | Recurrent Neural Networks | Slides Video Lecture |
D2L Sections 9, 10 DL Section 10 ISL Section 10.5 Voelker et al., 2019 Andrej Karpathy's "The Unreasonable Effectiveness of RNNs" |
| 16 | 12/11/2024 | Attention Mechanisms | Slides Video Lecture |
D2L Section 11 PML Section 15.4-15.7 Bahdanau et al., 2015 Vaswani et al., 2018 Radford et al., 2018 Radford et al.s, 2019 Lilian Weng's "The Transformer Family" |
| 17 | 14/11/2024 | Advanced Optimization | Slides Demo Video Lecture |
DL Section 8 D2L Section 12 |
| 18 | 19/11/2024 | Variational Autoencoders and Normalizing Flows | Slides Video Lecture |
DL Section 14, 20.10.3 PML Section 19.3.6, 20.3 PML2 Section 20, 21, 23 |
| 19 | 21/11/2024 | Generative Adversarial Networks and Diffusion Models | Slides Video Lecture |
D2L Section 20 DL Section 20.10.4 PML2 Section 25, 26 Goodfellow et al., 2014 Ho et al., 2020 |
| 20 | 26/11/2024 | Robustness (Guest Lecture from Dr. P Michael Furlong) |
Slides Video Lecture |
Goodfellow et al., 2014 Wiyatno, 2019 OpenAI, 2021 |
| 21 | 28/11/2024 | Differential Privacy (Guest Lecture from Saber Malekmohammadi) |
Slides Video Lecture |
|
| 22 | 03/12/2024 | Fairness (Guest Lecture from Dr. Terrence C. Stewart) |
Slides Video Lecture |
The four assignments will be posted here.
| ASSIGNMENT | MATERIALS | Posted | Due Date |
|---|---|---|---|
| 1 | Assignment 1 | 11 September 2024 | 27 September 2024 |
| 2 | Assignment 2 | 30 September 2024 | 21 October 2024 |
| 3 | Assignment 3 | 23 October 2024 | 8 November 2024 |
| 4 | Assignment 4 | 11 November 2024 | 29 November 2024 |