# CS229 2018 Lecture Notes > Free, unofficial lecture notes for Stanford CS229: Machine Learning, written from Andrew Ng's Autumn 2018 lectures. Every lecture of the course has a chapter of continuous prose with the board derivations written out in full, links to the moment in the YouTube recording where each section is taught, and pointers to the official class notes (cs229-notes1 to cs229-notes13) where they go further. The notes cover supervised learning (linear and logistic regression, generalized linear models, Gaussian discriminant analysis, naive Bayes, support vector machines and kernels), generalization (bias and variance, regularization, cross-validation, learning theory and VC dimension), decision trees, ensembles and neural networks, practical advice for debugging learning algorithms, unsupervised learning (k-means, the EM algorithm, factor analysis, PCA, ICA) and reinforcement learning (MDPs, value and policy iteration, fitted value iteration, LQR, policy search). Each link below is the chapter as Markdown, with the maths in TeX; the same chapter as a web page is at the same path ending in .html. ## Supervised Learning - [1. Introduction](https://raimbekovm.github.io/cs229-2018-notes/lectures/01-introduction.md): What machine learning is, from Samuel's checkers program to Mitchell's experience–task–performance definition, and the five areas CS229 covers: supervised learning, learning strategy, deep learning, unsupervised learning and reinforcement learning. - [2. Linear Regression and Gradient Descent](https://raimbekovm.github.io/cs229-2018-notes/lectures/02-linear-regression.md): Linear regression fitted two ways: by gradient descent on the least-squares cost — batch and stochastic, with Ng's rules for the learning rate — and in one step by the normal equations, derived with matrix derivatives and the trace. - [3. Locally Weighted and Logistic Regression](https://raimbekovm.github.io/cs229-2018-notes/lectures/03-locally-weighted-logistic.md): Locally weighted regression as the course's first non-parametric algorithm, least squares derived as maximum likelihood under Gaussian noise, logistic regression for binary classification fitted by gradient ascent, and Newton's method as the faster alternative when the parameters are few. - [4. Perceptron and Generalized Linear Models](https://raimbekovm.github.io/cs229-2018-notes/lectures/04-perceptron-glm.md): The perceptron as logistic regression with a hard threshold, the exponential family of distributions and why its log-partition function hands over the mean and variance by differentiation, generalized linear models as a recipe that turns any member into a regression model with one shared update rule, and softmax regression for multi-class classification as cross-entropy minimisation. - [5. GDA and Naive Bayes](https://raimbekovm.github.io/cs229-2018-notes/lectures/05-gda-naive-bayes.md): Generative learning algorithms, which model p(x | y) and p(y) and classify by Bayes' rule: Gaussian discriminant analysis and why it implies a logistic posterior but not conversely, when its stronger assumptions beat logistic regression and when they cost it, and naive Bayes for spam filtering with binary word features. - [6. Support Vector Machines](https://raimbekovm.github.io/cs229-2018-notes/lectures/06-support-vector-machines.md): Laplace smoothing and why it is the Bayesian answer to a zero count, the multinomial event model that lets naive Bayes count repeated words, the case for building something quick and dirty first, and the start of support vector machines: functional and geometric margins and the optimisation problem whose solution is the maximum-margin separator. - [7. Kernels](https://raimbekovm.github.io/cs229-2018-notes/lectures/07-kernels.md): How the optimal margin classifier becomes the support vector machine: the rescaling that turns the margin problem into minimising ½‖w‖², the representer theorem that writes w as a combination of training examples, the dual in which data appear only through inner products, kernels that compute those inner products in feature spaces too large to write down, Mercer's condition for which functions qualify, and the soft margin for data that will not separate. ## Generalization - [8. Data Splits, Models and Cross-Validation](https://raimbekovm.github.io/cs229-2018-notes/lectures/08-cross-validation.md): How to tell an algorithm that underfits from one that overfits, and what to do about each: bias and variance on the housing example, regularisation as a penalty λ‖θ‖² and as a Gaussian prior on θ, train/dev/test splits and how large each should be, k-fold and leave-one-out cross-validation for small datasets, and forward search for feature selection. - [Learning Theory (discussion section)](https://raimbekovm.github.io/cs229-2018-notes/lectures/08b-learning-theory.md): Why low training error says anything about test error: the learning algorithm as an estimator with a sampling distribution, bias and variance as its first two moments, the split of test error into Bayes, approximation and estimation error, empirical risk minimisation, Hoeffding's inequality and the union bound, uniform convergence and sample complexity for a finite hypothesis class, the guarantee ε(ĥ) ≤ ε(h*) + 2γ, and the VC dimension that extends it to infinite classes. ## Trees and Neural Networks - [9. Decision Trees and Ensemble Methods](https://raimbekovm.github.io/cs229-2018-notes/lectures/09-decision-trees-ensembles.md): Decision trees as greedy recursive partitioning of the input space, why splits are chosen by a strictly concave loss such as cross-entropy rather than misclassification, regression trees and categorical features, how trees are regularised and why they are fast, their weakness on additive structure — and how ensembles fix them: the variance of an average of correlated models, bagging, random forests, and boosting with AdaBoost. - [10. Introduction to Neural Networks](https://raimbekovm.github.io/cs229-2018-notes/lectures/10-neural-networks.md): Neural networks built up from logistic regression: one neuron as a linear part plus an activation, three independent neurons for multi-label outputs against a softmax for mutually exclusive classes and why its gradient is still ŷ − y, regression outputs, hidden layers and what fully connected means, forward propagation with the shape of every matrix, vectorising over a batch with broadcasting, and why backpropagation starts at the last layer. - [11. Backprop and Improving Neural Networks](https://raimbekovm.github.io/cs229-2018-notes/lectures/11-backprop.md): Backpropagation derived by hand for a three-layer network, with shape analysis to place every transpose and the caching that makes it cheap; then what makes training work: sigmoid, tanh and ReLU and why a network needs a non-linearity, input normalisation, vanishing and exploding gradients, variance-preserving initialisation (1/n, He's 2/n, Glorot's 2/(n_in + n_out)) and symmetry breaking, mini-batch gradient descent, and momentum. ## Applying Machine Learning - [12. Debugging ML Models and Error Analysis](https://raimbekovm.github.io/cs229-2018-notes/lectures/12-debugging-ml.md): How to decide what to do next when a learning algorithm does not work well enough: learning curves that separate high variance from high bias, the test that tells a failing optimiser from a wrong objective, the same reasoning applied to an autonomous helicopter, error analysis that credits each stage of a pipeline by plugging in ground truth, and ablative analysis that credits each component by removing it. ## Unsupervised Learning - [13. Expectation-Maximization Algorithms](https://raimbekovm.github.io/cs229-2018-notes/lectures/13-em-algorithms.md): Unsupervised learning from unlabelled data: k-means clustering and why it converges, anomaly detection by density estimation, the mixture of Gaussians as GDA with the labels hidden, the EM algorithm for fitting it as k-means with soft assignments, Jensen's inequality, and the general derivation of EM as repeatedly building and maximising a lower bound on the log-likelihood. - [14. EM Algorithm and Factor Analysis](https://raimbekovm.github.io/cs229-2018-notes/lectures/14-factor-analysis.md): How the general EM algorithm turns into concrete updates for the mixture of Gaussians, EM as coordinate ascent on the lower bound, why a Gaussian cannot be fitted when there are fewer examples than dimensions, the factor analysis model for high-dimensional data from few examples, marginals and conditionals of Gaussians, and EM for factor analysis. - [15. PCA and ICA](https://raimbekovm.github.io/cs229-2018-notes/lectures/15-pca-ica.md): Principal components analysis as the direction of maximum variance and, equivalently, of minimum reconstruction error; why the answer is the top eigenvectors of the covariance; centring, scaling and choosing k; where PCA helps and where it is misused; a map of the unsupervised algorithms; and the cocktail-party problem that independent components analysis solves, with its ambiguities. ## Reinforcement Learning - [16. Independent Component Analysis and RL](https://raimbekovm.github.io/cs229-2018-notes/lectures/16-ica-rl.md): Why ICA cannot unmix Gaussian sources, how a density changes under a linear map, the sigmoid-CDF source prior and the maximum-likelihood gradient rule for the unmixing matrix, where ICA is used and where it breaks; then reinforcement learning: the credit assignment problem, Markov decision processes on an 11-state gridworld, discounted payoff and the optimal policy. - [17. MDPs and Value/Policy Iteration](https://raimbekovm.github.io/cs229-2018-notes/lectures/17-value-policy-iteration.md): The value function of a policy and Bellman's equations, the optimal value function and how the optimal policy is read off it, value iteration with synchronous and asynchronous updates and how fast it converges, policy iteration, estimating the transition probabilities from experience, and exploration versus exploitation with epsilon-greedy and Boltzmann exploration. - [18. Continuous State MDPs and Model Simulation](https://raimbekovm.github.io/cs229-2018-notes/lectures/18-continuous-mdps.md): How to solve an MDP whose state is a vector of real numbers: discretization and the curse of dimensionality, building a simulator from physics or by regressing the next state on the current state and action, fitted value iteration with linear regression on features of the state, why it carries no convergence guarantee, and how to turn the fitted value function into a policy fast enough to fly a helicopter. - [19. Finite-Horizon MDPs and the Linear Quadratic Regulator](https://raimbekovm.github.io/cs229-2018-notes/lectures/19-lqr.md): Two generalizations of the MDP, rewards that depend on the action and a finite horizon with a non-stationary optimal policy solved by backward dynamic programming, then the linear quadratic regulator: linear dynamics from regression or linearization, a quadratic reward, an exactly quadratic value function and linear optimal policy from the Riccati recursion, and why the noise does not change that policy — with the sign errors in the official notes' equations corrected. - [20. RL Debugging and Direct Policy Search](https://raimbekovm.github.io/cs229-2018-notes/lectures/20-policy-search.md): The last lecture of CS229: the helicopter diagnostics restated in reinforcement-learning notation, then direct policy search, which skips the value function and tunes a stochastic policy by gradient ascent on its expected payoff. The REINFORCE update, a derivation that its average is exactly that gradient, why its noise makes it slow and how weighting each action only by later rewards and subtracting a baseline cut that noise without changing the average, why policy search copes with partial observability, and when a policy is simpler to learn than a value function. ## Optional - [Every chapter in one file](https://raimbekovm.github.io/cs229-2018-notes/llms-full.txt): the whole book as Markdown - [The book as a PDF](https://raimbekovm.github.io/cs229-2018-notes/CS229-2018-Lecture-Notes.pdf) - [The 2018 lecture videos](https://www.youtube.com/playlist?list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU): Stanford Online's YouTube playlist - [Official course site](https://cs229.stanford.edu/): Stanford CS229 - [Source repository](https://github.com/raimbekovm/cs229-2018-notes): corrections are welcome as issues or pull requests