Resources

This page contains some resources that may be useful to you, and they can serve as supplements to the lectures, discussions, and homeworks for this semester.

Textbook

Deep Learning: Foundations and Concepts by Christopher M. Bishop and Hugh Bishop.

Notes

Below are lecture notes from Anant Sahai’s iteration of the course:

• Note 2 - Linear Regression
• Note 3 - Feature Engineering
• Note 4 - MLE and MAP for Regression
• Note 5 - Bias-Variance Tradeoff
• Note 6 - Multivariate Gaussians
• Note 7 - MLE and MAP Part 2
• Note 8 - Kernels and Ridge Regression
• Note 9 - Total Least Squares
• Note 10 - PCA
• Note 11 - CCA
• Note 12 - Optimization
• Note 13 - Gradient Descent
• Note 14 - Neural Networks
• Note 15 - Training Neural Networks
• Note 16 - Generative vs. Discriminative Classification
• Note 18 - Gaussian Discriminant Analysis
• Note 19 - Clustering
• Note 20 - Support Vector Machines
• Note 21 - Generalization and Stability
• Note 22 - Duality
• Note 25 - Decision Trees
• Note 26 - Boosting
• Note 27 - Convolutional Neural Networks

Other Notes

• Notes from Jonathan Shewchuk’s iteration of the course
• Review Session Slides from Chandan Singh
• Mathematics for Machine Learning by Garrett Thomas
• Lecture slides and code from Josh Tobin
• Notes and Practice from Alvin Wan
• Discussion Slides from Arvind Rajaraman

Tips

These tips have been collected through the years from professors, past and present. You can also check out the Learning How To Learn Coursera course for other general tips.

Don’t fall behind

In a conceptual class such as this, it is particularly important to maintain a steady effort throughout the semester, rather than hope to cram just before homework deadlines or exams. This is because it takes time and practice for the ideas to sink in. Make sure you allocate a sufficient number of hours every week to the class, including enough time for understanding the material as well as for doing assignments.

Take the homeworks seriously

The homeworks are explicitly designed to help you to learn the material as you go along. Read the solutions carefully. You may well learn a different way of looking at the problem, and you may also benefit from emulating the style of the solutions.

Don’t procrastinate on homework

Our best advice is to read through the homework problems as soon as they are available. Do not wait until the night before it is due to start working on the homework.

Make use of office hours

The instructor and TAs hold office hours expressly to help you. You are free to attend as many office hours as you wish (you are not constrained just to use the office hours of your section TA). You will also likely get more out of an office hour if you have spent a little time in advance thinking about the questions you have. Be prepared with copies of your lecture notes so you can point out which parts of lecture you believe may be relevant to your question.

Take part in discussion sections

Discussion sections are not auxiliary lectures. They are an opportunity for interactive learning, through guided group problem solving and other activities. The success of a discussion section depends largely on the willingness of students to participate actively in it. As with office hours, the better prepared you are for the discussion, the more you are likely to get out of it. Do not come to discussion looking for a way to catch up with lecture.