# Introduction to Machine Learning

**University of California, Berkeley**, Fall 2024

Welcome to CS 189/289A! This class covers theoretical foundations, algorithms, methodologies, and applications for machine learning. Topics may include supervised methods for regression and classication (linear models, trees, neural networks, ensemble methods, instance-based methods); generative and discriminative probabilistic models; deep learning models including CNNs, transformers, graph neural networks for vision and language tasks; and Markovian models for reinforcement learning and robotics.

**Textbook:** *Deep Learning* by Bishop and Bishop.

Professors will post slides prior to lecture at this Google Drive folder (for faster access). The material here is redundant with the website, but it may take up to a day or two for the website to get updated with the slides after lecture. The “Post-Lecture” subfolder contains updates to slides that the professors may make right after lecture. In addition, lecture recordings will be uploaded automatically to this Haas Panopto folder.

If you have not been added to **EdStem**, you may join through this link. The **Gradescope** code is **DKPWZW**.

**Note:** The topics for future lectures, discussions, and HWs are **tentative** and may be moved around, changed, or removed.

## Overview

### Week 1

8/29 | Lecture 1 Introduction and Logistics (Slides) | Reading 1.1-1.2.4 |

Homework 1 Math Review Due 9/12 11:59pm) | PDF / Files / LaTeX Template |

### Week 2

9/3 | Lecture 2 Maximum Likelihood Estimation (Slides) | Reading 2-2.1.2 (rules of probability: sum, product) 2.1.6 (independent RVs) 2.2-2.2.1 (probability densities in continuous spaces) 2.3-2.3.2 (univariate Gaussian, likelihood) 3-3.1.3 (Bernoulli, binomial, multinomial, MLE) |

Discussion 0 Math Pre-Requisites Review | Worksheet / Solutions | |

9/5 | Lecture 3 Multivariate Gaussians (Slides) | Reading 3-3.2.3 (multivariate Gaussians: geometry, moments, covariance forms) |

### Week 3

9/10 | Lecture 4 Linear Regression 1 (Slides) | Reading |

Discussion 1 MLE & Gaussians | Worksheet / Solutions | |

9/12 | Lecture 5 Linear Regression 2 (Slides) | Reading 1.25-1.26 (regularization, model selection) 2.1.3 (Bayes theorem) 2.6 (Bayesian modeling) 4.1.6 (regularization for linear regression) 9.2.2 (Lasso regularization) |

Homework 2 Linear/Logistic Regression & Classification Due 9/25 11:59pm) | PDF / Files / LaTeX Template |

### Week 4

9/17 | Lecture 6 Classification - Generative & Discriminative (Slides) | Reading 5.1, 5.2.1, 5.2.2, 5.2.4, 5.3 |

Discussion 2 Regularization & MAP | Worksheet / Solutions | |

9/19 | Lecture 7 Logistic Regression & Neural Networks (Slides) | Reading 5.3.1, 5.4.1-5.4.4, 6.1-6.2 |

### Week 5

9/24 | Lecture 8 Backpropagation and Gradient Descent 1 (Slides) | Reading 6.2, 6.4, 7.1, 7.2, 5.4.4 |

Discussion 3 Classification & Logistic Regression | Worksheet / Solutions | |

9/26 | Lecture 9 Backpropagation and Gradient Descent 2 (Slides) | Reading 5.4.4, 8 |

Homework 3 Neural Networks Due 10/9 11:59pm) | PDF / Files / LaTeX Template |

### Week 6

10/1 | Lecture 10 Neural Networks - CNNs, Batch Norm, & ResNets (Slides) | Reading 7.4, 9.5, 10 |

Discussion 4 Neural Networks and Backpropagation | Worksheet / Solutions | |

10/3 | Lecture 11 Neural Networks - Attention & Transformers (Slides) | Reading 5.3, 12.1 |

### Week 7

10/8 | Lecture 12 Dimensionality Reduction & PCA | Reading 16.1 |

Discussion 5 Convolution and Attention | Worksheet | |

10/10 | Lecture 13 t-SNE | |

Homework 4 Dimensionality Reduction & Decision Theory Due 10/23 11:59pm) |

### Week 8

10/15 | Lecture 14 Clustering | |

Discussion 6 Dimensionality Reduction Techniques | ||

10/16 | Midterm (7-9pm, Location TBD) | |

10/17 | Lecture 15 Multiway Classification, Decision Theory, & Model Evaluation |

### Week 9

10/22 | Lecture 16 Nearest Neighbors & Metric Learning | |

Discussion 7 Clustering and Decision Theory | ||

10/24 | Lecture 17 Decision Trees & Ensembling | |

Homework 5 Bias/Variance, Nearest Neighbors, Decision Trees Due 11/6 11:59pm) |

### Week 10

10/29 | Lecture 18 Bias-Variance Tradeoff & Over/Under-Fitting | |

Discussion 8 Bias/Variance and Nearest Neighbors | ||

10/31 | Lecture 19 Hidden Markov Models & Graphical Models 1 |

### Week 11

11/5 | Lecture 20 Hidden Markov Models & Graphical Models 2 | |

Discussion 9 Decision Trees and HMMs Intro | ||

11/7 | Lecture 21 Markov Decision Processes | |

Homework 6 Markovian Models & Reinforcement Learning Due 11/20 11:59pm) |

### Week 12

11/12 | Lecture 22 Reinforcement Learning | |

Discussion 10 HMMs Advanced and MDPs | ||

11/14 | Lecture 23 Robotics and Machine Learning |

### Week 13

11/19 | Lecture 24 Graph Neural Networks & Rotational Equivariance 1 | |

Discussion 11 MDPs & Reinforcement Learning | ||

11/21 | Lecture 25 Graph Neural Networks & Rotational Equivariance 2 | |

Homework 7 Graph Neural Networks & Applications of Deep Learning Due 12/4 11:59pm) |

### Week 14

11/26 | Lecture 26 Language and Vision Applications | |

11/28 | No Lecture (Thanksgiving) |

### Week 15

12/3 | Lecture 27 Special Topics - Causality | |

Discussion 12 Graph Neural Networks | ||

12/5 | Lecture 28 Special Topics - Computational Biology |

### Week 16 (RRR Week)

No lectures or discussions this week |

### Week 17 (Finals Week)

12/17 | Final Exam (8-11am, location TBD) |