Mathematics for Machine Learning

What you will learn

• Work with vectors, matrices and their decompositions
• Differentiate the functions models are built from
• Reason about uncertainty with probability and statistics

Syllabus

132 lessons in 13 sections. Take them in order, or open any lesson directly.

1. Mathematics for Machine Learning13 min
2. Chapter 0: Getting Ready7 lessons

   1. Getting Ready Overview4 min
   2. Notation and Symbols14 min, 5 exercises
   3. Sums, Products, and Set Notation16 min, 5 exercises
   4. Functions, Limits, and Continuity12 min, 5 exercises
   5. Single-Variable Calculus Refresher14 min, 5 exercises
   6. Complex Numbers in One Page13 min, 5 exercises
   7. NumPy for Mathematics12 min, 5 exercises
3. Chapter 1: Introduction and Motivation3 lessons

   1. Introduction and Motivation Overview3 min
   2. Finding Words for Intuitions17 min, 5 exercises
   3. Two Ways to Read This Book15 min, 5 exercises
4. Chapter 2: Linear Algebra11 lessons

   1. Linear Algebra Overview6 min
   2. Systems of Linear Equations12 min, 5 exercises
   3. Matrices15 min, 5 exercises
   4. Solving Systems of Linear Equations16 min, 5 exercises
   5. Vector Spaces16 min, 5 exercises
   6. Linear Independence16 min, 5 exercises
   7. Basis and Rank16 min, 5 exercises
   8. Linear Mappings18 min, 5 exercises
   9. Affine Spaces15 min, 5 exercises
   10. Chapter 2 Exercises and Solutions25 min, 5 exercises
   11. Chapter 2 Formula Sheet16 min
5. Chapter 3: Analytic Geometry12 lessons

   1. Analytic Geometry Overview6 min
   2. Norms13 min, 5 exercises
   3. Inner Products16 min, 5 exercises
   4. Lengths and Distances14 min, 5 exercises
   5. Angles and Orthogonality12 min, 5 exercises
   6. Orthonormal Basis13 min, 5 exercises
   7. Orthogonal Complement13 min, 5 exercises
   8. Inner Product of Functions13 min, 5 exercises
   9. Orthogonal Projections18 min, 5 exercises
   10. Rotations15 min, 5 exercises
   11. Chapter 3 Exercises and Solutions19 min, 5 exercises
   12. Chapter 3 Formula Sheet13 min
6. Chapter 4: Matrix Decompositions10 lessons

   1. Matrix Decompositions Overview8 min
   2. Determinant and Trace16 min, 5 exercises
   3. Eigenvalues and Eigenvectors18 min, 5 exercises
   4. Cholesky Decomposition14 min, 5 exercises
   5. Eigendecomposition and Diagonalization16 min, 5 exercises
   6. Singular Value Decomposition20 min, 5 exercises
   7. Matrix Approximation19 min, 5 exercises
   8. Matrix Phylogeny17 min, 5 exercises
   9. Chapter 4 Exercises and Solutions19 min, 5 exercises
   10. Chapter 4 Formula Sheet10 min
7. Chapter 5: Vector Calculus11 lessons

   1. Vector Calculus Overview9 min
   2. Differentiation of Univariate Functions14 min, 5 exercises
   3. Partial Differentiation and Gradients16 min, 5 exercises
   4. Gradients of Vector-Valued Functions15 min, 5 exercises
   5. Gradients of Matrices16 min, 5 exercises
   6. Useful Identities for Computing Gradients16 min, 5 exercises
   7. Backpropagation and Automatic Differentiation15 min, 5 exercises
   8. Higher-Order Derivatives21 min, 5 exercises
   9. Linearization and Multivariate Taylor Series18 min, 5 exercises
   10. Chapter 5 Exercises and Solutions19 min, 5 exercises
   11. Chapter 5 Formula Sheet17 min
8. Chapter 6: Probability and Distributions10 lessons

   1. Probability and Distributions Overview10 min
   2. Construction of a Probability Space17 min, 5 exercises
   3. Discrete and Continuous Probabilities18 min, 5 exercises
   4. Sum Rule, Product Rule, and Bayes Theorem15 min, 5 exercises
   5. Summary Statistics and Independence20 min, 5 exercises
   6. Gaussian Distribution15 min, 5 exercises
   7. Conjugacy and the Exponential Family19 min, 5 exercises
   8. Change of Variables and the Inverse Transform16 min, 5 exercises
   9. Chapter 6 Exercises and Solutions16 min, 5 exercises
   10. Chapter 6 Formula Sheet15 min
9. Chapter 7: Continuous Optimization9 lessons

   1. Continuous Optimization Overview6 min
   2. Optimization Using Gradient Descent19 min, 5 exercises
   3. Momentum and Stochastic Gradient Descent18 min, 5 exercises
   4. Constrained Optimization and Lagrange Multipliers20 min, 5 exercises
   5. Convex Sets and Convex Functions22 min, 5 exercises
   6. Linear and Quadratic Programming16 min, 5 exercises
   7. Legendre-Fenchel Transform and Convex Conjugate20 min, 5 exercises
   8. Chapter 7 Exercises and Solutions21 min, 5 exercises
   9. Chapter 7 Formula Sheet15 min
10. Recall Drill3 min
11. Chapter 8: When Models Meet Data11 lessons

    1. When Models Meet Data Overview8 min
    2. Data, Models, and Learning19 min, 5 exercises
    3. Empirical Risk Minimization16 min, 5 exercises
    4. Regularization and Cross-Validation19 min, 5 exercises
    5. Maximum Likelihood Estimation17 min, 5 exercises
    6. MAP Estimation and Model Fitting19 min, 5 exercises
    7. Probabilistic Modeling and Inference23 min, 5 exercises
    8. Directed Graphical Models23 min, 5 exercises
    9. Model Selection30 min, 5 exercises
    10. Chapter 8 Worked Problems15 min, 5 exercises
    11. Chapter 8 Formula Sheet17 min
12. Chapter 9: Linear Regression12 lessons

    1. Linear Regression Overview8 min
    2. Problem Formulation18 min, 5 exercises
    3. Maximum Likelihood Estimation for Linear Regression21 min, 5 exercises
    4. Overfitting in Linear Regression22 min, 5 exercises
    5. MAP Estimation and Regularization23 min, 5 exercises
    6. Bayesian Linear Regression18 min, 5 exercises
    7. The Parameter Posterior18 min, 5 exercises
    8. Posterior Predictions16 min, 5 exercises
    9. Computing the Marginal Likelihood17 min, 5 exercises
    10. Maximum Likelihood as Orthogonal Projection16 min, 5 exercises
    11. Chapter 9 Worked Problems12 min, 5 exercises
    12. Chapter 9 Formula Sheet14 min
13. Chapter 10: Dimensionality Reduction with PCA11 lessons

    1. Dimensionality Reduction Overview9 min
    2. Problem Setting12 min, 5 exercises
    3. The Maximum Variance Perspective11 min, 5 exercises
    4. The Projection Perspective9 min, 5 exercises
    5. Finding the Principal Subspace9 min, 5 exercises
    6. Eigenvector Computation and Low-Rank Approximations10 min, 5 exercises
    7. PCA in High Dimensions8 min, 5 exercises
    8. Key Steps of PCA in Practice9 min, 5 exercises
    9. The Latent Variable Perspective on PCA9 min, 5 exercises
    10. Chapter 10 Worked Problems11 min, 5 exercises
    11. Chapter 10 Formula Sheet11 min
14. Chapter 11: Density Estimation with GMM11 lessons

    1. Density Estimation Overview12 min
    2. The Gaussian Mixture Model8 min, 5 exercises
    3. Maximum Likelihood and Its Obstacle8 min, 5 exercises
    4. Responsibilities10 min, 5 exercises
    5. Updating the Means9 min, 5 exercises
    6. Updating the Covariances8 min, 5 exercises
    7. Updating the Mixture Weights7 min, 5 exercises
    8. The EM Algorithm8 min, 4 exercises
    9. The Latent Variable Perspective on Mixtures8 min, 5 exercises
    10. Chapter 11 Worked Problems20 min, 5 exercises
    11. Chapter 11 Formula Sheet17 min
15. Chapter 12: Classification with Support Vector Machines12 lessons

    1. Classification Overview11 min
    2. Separating Hyperplanes10 min, 3 exercises
    3. The Concept of the Margin6 min, 3 exercises
    4. Why the Margin Can Be Set to One8 min, 3 exercises
    5. The Soft Margin SVM10 min, 3 exercises
    6. The Hinge Loss11 min, 3 exercises
    7. The Dual Support Vector Machine9 min, 3 exercises
    8. The Convex Hull View7 min, 3 exercises
    9. Kernels7 min, 3 exercises
    10. Numerical Solution9 min, 3 exercises
    11. Chapter 12 Worked Problems11 min, 5 exercises
    12. Chapter 12 Formula Sheet16 min