CS229 2018 Lecture Notes
  • PDF
  1. CS229 2018 Lecture Notes
  • CS229 2018 Lecture Notes
  • Supervised Learning
    • 1  Introduction
    • 2  Linear Regression and Gradient Descent
    • 3  Locally Weighted and Logistic Regression
    • 4  Perceptron and Generalized Linear Models
    • 5  GDA and Naive Bayes
    • 6  Support Vector Machines
    • 7  Kernels
  • Generalization
    • 8  Data Splits, Models and Cross-Validation
    • Learning Theory
  • Trees and Neural Networks
    • 9  Decision Trees and Ensemble Methods
    • 10  Introduction to Neural Networks
    • 11  Backprop and Improving Neural Networks
  • Applying Machine Learning
    • 12  Debugging ML Models and Error Analysis
  • Unsupervised Learning
    • 13  Expectation-Maximization Algorithms
    • 14  EM Algorithm and Factor Analysis
    • 15  PCA and ICA
  • Reinforcement Learning
    • 16  Independent Component Analysis and RL
    • 17  MDPs and Value/Policy Iteration
    • 18  Continuous State MDPs and Model Simulation
    • 19  Finite-Horizon MDPs and the Linear Quadratic Regulator
    • 20  RL Debugging and Direct Policy Search

CS229 2018 Lecture Notes

Machine Learning — Stanford University, Autumn 2018

Lecture notes written from the Stanford CS229 Machine Learning videos (Andrew Ng, Autumn 2018): the board derivations, the intuition given out loud, and a timestamp into the recording for every section.
Author

Murat Raimbekov

CS229 2018 Lecture Notes

CS229 lecture notes · Stanford · Autumn 2018 · Unofficial

Andrew Ng’s CS229,
written up from the board.

Linear regression to reinforcement learning — the derivations worked at the blackboard, the intuition given out loud, and a timestamp into the 2018 recording for every section.

Start with lecture 1 Read as PDF
cost contours · batch gradient descent · lecture 2
TipHow to read these notes

Each lecture is a standalone page written to be read straight through. The recording stays one click away: the chapter links its video under the title, and every section carries a timestamp that opens the lecture at the moment the idea is introduced, so you can watch the derivation being built when the prose is not enough. Sections, equations and figures are numbered and individually linkable.

Lectures

The course runs to twenty lectures and one discussion section, and every one has a chapter; the roadmap groups them by theme.

Lecture 1

Introduction

What machine learning is, the five areas the course covers, and the board pictures every later lecture is drawn on.

Complete

Lecture 2

Linear Regression and Gradient Descent

Fitting a hypothesis by least squares: batch and stochastic gradient descent, and the normal equations that skip iteration altogether.

Complete

Lecture 3

Locally Weighted and Logistic Regression

Least squares as maximum likelihood, a regression that refits around every query, and the logistic model for classification with Newton’s method.

Complete

Lecture 4

Perceptron and Generalized Linear Models

The perceptron, the exponential family, and the recipe that turns a choice of distribution into a learning algorithm, softmax regression included.

Complete

Lecture 5

GDA and Naive Bayes

Modelling how each class generates its inputs instead of the boundary between them: Gaussian discriminant analysis and naive Bayes.

Complete

Lecture 6

Support Vector Machines

Laplace smoothing and the event models for text, then the geometric margin and the optimisation problem that maximises it.

Complete

Lecture 7

Kernels

The dual of the margin problem, inner products in feature spaces you never compute, and the soft margin for data that will not separate.

Complete

Lecture 8

Data Splits, Models and Cross-Validation

Bias and variance, regularisation, and how to choose a model with data you did not train on.

Complete

Discussion section

Learning Theory

Why training error says anything about test error: the union bound, Hoeffding’s inequality, uniform convergence and the VC dimension.

Complete

Lecture 9

Decision Trees and Ensemble Methods

Greedy recursive splits, the losses that choose them, and the bagging, random forests and boosting that make trees competitive.

Complete

Lecture 10

Introduction to Neural Networks

Logistic regression as a single neuron, and what stacking them into layers buys.

Complete

Lecture 11

Backprop and Improving Neural Networks

Backpropagation worked out by hand, then initialisation, normalisation and the optimisers that make training work in practice.

Complete

Lecture 12

Debugging ML Models and Error Analysis

Diagnosing a learning algorithm that does not work: bias versus variance, optimisation versus objective, and error and ablative analysis.

Complete

Lecture 13

Expectation-Maximization Algorithms

Clustering with k-means, mixtures of Gaussians, and the EM algorithm that fits a model with latent variables.

Complete

Lecture 14

EM Algorithm and Factor Analysis

EM’s concrete updates and its view as coordinate ascent, and factor analysis for few examples in many dimensions.

Complete

Lecture 15

PCA and ICA

Projecting data onto the directions of greatest variance, when that helps and when it does not, and the cocktail-party problem ICA solves.

Complete

Lecture 16

Independent Component Analysis and RL

Finishing ICA by maximum likelihood, then Markov decision processes and the Bellman equations.

Complete

Lecture 17

MDPs and Value/Policy Iteration

Solving an MDP by value iteration and policy iteration, and learning its model from experience.

Complete

Lecture 18

Continuous State MDPs and Model Simulation

Discretization and its limits, learning a simulator from flight logs, and fitted value iteration for continuous states.

Complete

Lecture 19

Finite-Horizon MDPs and the Linear Quadratic Regulator

State-action rewards, finite horizons and non-stationary policies, linearization, and the linear quadratic regulator.

Complete

Lecture 20

RL Debugging and Direct Policy Search

The helicopter diagnostics in RL notation, stochastic policies, REINFORCE and its variance, partial observability.

Complete

What these notes are

The official CS229 notes are the formal account of the course, and a good one. What they cannot carry is the lecture itself: the order in which Andrew Ng motivates an idea before he formalises it, the picture he draws to make an update rule obvious, the question from the room that exposes the step everyone was about to skip.

These notes are written from the Autumn 2018 recordings rather than from the official notes, and they are meant to sit beside both. Each page aims to be:

  • Faithful to the lecture — it follows what was taught and why, and links each section to the moment in the video where it happens.
  • Complete in its mathematics — the derivations done at the board are written out in full, with every symbol defined.
  • A bridge, not a copy — where the official notes go further, the chapter says which section to read next rather than reproducing it.

The whole book is also a single PDF. For an AI assistant, llms.txt maps every chapter to a Markdown copy with the maths in TeX, and llms-full.txt holds the book in one file. The same treatment of Stanford’s deep learning for computer vision course is in the companion CS231n 2025 Lecture Notes.

Roadmap

  1. Supervised learning — linear and logistic regression, generalized linear models, generative learning, support vector machines and kernels
  2. Generalization — bias and variance, model selection, learning theory
  3. Trees and neural networks — decision trees and ensembles, neural networks and backpropagation
  4. Applying machine learning — debugging learning algorithms and error analysis
  5. Unsupervised learning — k-means, EM, factor analysis, PCA and ICA
  6. Reinforcement learning — MDPs, value and policy iteration, continuous states, LQR

Contributing

Found an error? Corrections are genuinely welcome — there is an Edit this page link at the bottom of every lecture, which opens a pull request in your browser. For anything larger, open an issue.

Disclaimer

This is an unofficial resource, not affiliated with Stanford University. For official course materials see cs229.stanford.edu and the 2018 lecture videos.

The notes are released under the MIT licence, which covers the writing and the diagrams made for this project. Any figure reproduced from elsewhere remains the property of its original authors and is credited in its caption — see NOTICE.

1  Introduction
  • Edit this page
  • Report an issue