Introduction to Machine Learning
University of California, Berkeley, Fall 2024
Welcome to CS 189/289A! This class covers theoretical foundations, algorithms, methodologies, and applications for machine learning. Topics may include supervised methods for regression and classication (linear models, trees, neural networks, ensemble methods, instance-based methods); generative and discriminative probabilistic models; deep learning models including CNNs, transformers, graph neural networks for vision and language tasks; and Markovian models for reinforcement learning and robotics.
Textbook: Deep Learning by Bishop and Bishop.
Professors will post slides prior to lecture at this Google Drive folder (for faster access). The material here is redundant with the website, but it may take up to a day or two for the website to get updated with the slides after lecture. The “Post-Lecture” subfolder contains updates to slides that the professors may make right after lecture. In addition, lecture recordings will be uploaded automatically to this Haas Panopto folder.
If you have not been added to EdStem, you may join through this link. The Gradescope code is DKPWZW.
Note: The topics for future lectures, discussions, and HWs are tentative and may be moved around, changed, or removed.
Overview
Week 1
| 8/29 | Lecture 1 Introduction and Logistics (Slides) | Reading 1.1-1.2.4 |
| Homework 1 Math Review | PDF / Files / LaTeX Template |
Week 2
| 9/3 | Lecture 2 Maximum Likelihood Estimation (Slides) | Reading 2-2.1.2 (rules of probability: sum, product) 2.1.6 (independent RVs) 2.2-2.2.1 (probability densities in continuous spaces) 2.3-2.3.2 (univariate Gaussian, likelihood) 3-3.1.3 (Bernoulli, binomial, multinomial, MLE) |
| Discussion 0 Math Pre-Requisites Review | Worksheet / Solutions | |
| 9/5 | Lecture 3 Multivariate Gaussians (Slides) | Reading 3-3.2.3 (multivariate Gaussians: geometry, moments, covariance forms) |
Week 3
| 9/10 | Lecture 4 Linear Regression 1 (Slides) | Reading |
| Discussion 1 MLE & Gaussians | Worksheet / Solutions | |
| 9/12 | Lecture 5 Linear Regression 2 (Slides) | Reading 1.25-1.26 (regularization, model selection) 2.1.3 (Bayes theorem) 2.6 (Bayesian modeling) 4.1.6 (regularization for linear regression) 9.2.2 (Lasso regularization) |
| Homework 2 Linear/Logistic Regression & Classification | PDF / Files / LaTeX Template |
Week 4
| 9/17 | Lecture 6 Classification - Generative & Discriminative (Slides) | Reading 5.1, 5.2.1, 5.2.2, 5.2.4, 5.3 |
| Discussion 2 Regularization & MAP | Worksheet / Solutions | |
| 9/19 | Lecture 7 Logistic Regression & Neural Networks (Slides) | Reading 5.3.1, 5.4.1-5.4.4, 6.1-6.2 |
Week 5
| 9/24 | Lecture 8 Backpropagation and Gradient Descent 1 (Slides) | Reading 6.2, 6.4, 7.1, 7.2, 5.4.4 |
| Discussion 3 Classification & Logistic Regression | Worksheet / Solutions | |
| 9/26 | Lecture 9 Backpropagation and Gradient Descent 2 (Slides) | Reading 5.4.4, 8 |
| Homework 3 Neural Networks | PDF / Files / LaTeX Template |
Week 6
| 10/1 | Lecture 10 Neural Networks - CNNs, Batch Norm, & ResNets (Slides) | Reading 7.4, 9.5, 10 |
| Discussion 4 Neural Networks and Backpropagation | Worksheet / Solutions | |
| 10/3 | Lecture 11 Neural Networks - Attention & Transformers (Slides) | Reading 5.3, 12.1 |
Week 7
| 10/8 | Lecture 12 Dimensionality Reduction & PCA (Slides) | Reading 16.1 |
| Discussion 5 Convolution and Attention | Worksheet / Solutions | |
| 10/10 | Lecture 13 t-SNE (Slides) | |
| Homework 4 Dimensionality Reduction & Decision Theory | PDF / Files / LaTeX Template |
Week 8
| 10/15 | Lecture 14 Clustering (Slides) | Reading 15.1 (not including 15.1.1), 15.2 |
| Discussion 6 Dimensionality Reduction Techniques | Worksheet / Solutions | |
| 10/16 | Midterm (7-9pm, Dwinelle 155) | |
| 10/17 | Lecture 15 Nearest Neighbors & Metric Learning (Slides) | Reading 3.5, 6.3.5 |
Week 9
| 10/22 | Lecture 16 Model Evaluation (Slides) | Reading 5.25, 5.26 |
| Discussion 7 Gaussian Mixture Models | Worksheet / Solutions | |
| 10/24 | Lecture 17 Decision Trees & Ensembling (Slides) | Reading 9.6 not including 9.6.1 (model averaging) |
| Homework 5 Bias/Variance, Nearest Neighbors, Decision Trees | PDF / Files / LaTeX Template |
Week 10
| 10/29 | Lecture 18 Bias-Variance Tradeoff & Over/Under-Fitting (Slides) | Reading 4.2, 4.3, 9.3.2 |
| Discussion 8 Bias/Variance, Decision Trees, & Nearest Neighbors | Worksheet / Solutions | |
| 10/31 | Lecture 19 Hidden Markov Models & Graphical Models 1 (Slides) | Reading 11 |
Week 11
| 11/5 | Lecture 20 Hidden Markov Models & Graphical Models 2 (Slides) | Reading Bishop 11; Barber 23.2.2, 23.2.5 |
| Discussion 9 Random Forests and HMMs Intro | Worksheet / Solutions | |
| 11/7 | Lecture 21 Generative Models 1 (Slides) | Reading 14.3 |
| Homework 6 Graphical Models & Langevin MCMC | PDF / Files / LaTeX Template |
Week 12
| 11/12 | Lecture 22 Generative Models 2 (Slides) | Reading 20.3 |
| Discussion 10 Langevin Sampling | Worksheet / Solutions | |
| 11/14 | Lecture 23 Markov Decision Processes & Reinforcement Learning (Slides) |
Week 13
| 11/19 | Lecture 24 Graph Neural Networks 1 (Slides) | Reading 13 |
| Discussion 11 Score Matching; MDPs & Reinforcement Learning | Worksheet / Solutions | |
| 11/21 | Lecture 25 Graph Neural Networks 2 (Slides) | Reading 13 |
| Homework 7 Score Matching, MDPs, Graph Neural Networks & Kernels | PDF / Files / LaTeX Template |
Week 14
| 11/26 | Lecture 26 Kernel Methods (Slides) | |
| 11/28 | No Lecture (Thanksgiving) |
Week 15
| 12/3 | Lecture 27 Causality (Slides) | Optional Reading Hardt Chapters 9 & 10 |
| Discussion 12 Graph Neural Networks, Kernel Methods & Causality | Worksheet / Solutions | |
| 12/5 | Lecture 28 Special Topics - Computational Biology (Slides) |
Week 16 (RRR Week)
| 12/10 | TA Review Session Pre-Midterm Material (1-3 PM, Soda 306) | Slides |
| 12/11 | TA Review Session Post-Midterm Material (9-11 AM, Soda Wozniak Lounge) | Slides |