Overview

This course covers linear algebra, calculus, probability theory, statistics, and high-dimensional spaces for machine learning. The learning outcomes include understanding key mathematical concepts essential for machine learning applications. Students will learn skills such as matrix operations, eigenvalues, differentiation, integration, probability measures, random variables, hypothesis testing, and optimization techniques. The teaching method involves theoretical explanations, proofs, and practical examples. This course is intended for individuals interested in gaining a strong mathematical foundation for machine learning algorithms and applications.

Syllabus

Welcome!.
(A) Linear algebra 1: Vector spaces.
(A) Linear algebra 2: Basis and dimension.
(A) Linear algebra 3: Direct sum.
(A) Linear algebra 4: Linear maps, kernel, range.
(A) Linear algebra 5: Matrices.
(A) Linear algebra 6: Invertible maps and matrices.
(A) Linear algebra 7: Transpose.
(A) Linear algebra 8: Change of basis.
(A) Linear algebra 9: Rank of a matrix.
(A) Linear algebra 10: Quotient space; equivalence relation.
(A) Linear algebra 11: Determinant.
(A) Linear algebra 12: Eigenvalues.
(A) Linear algebra 13: Characteristic polynomial.
(A) Linear algebra 14: Trace of a matrix.
(A) Linear algebra 15: Diagonalization; triangular matrices.
(A) Linear algebra 16: Metric spaces.
(A) Linear algebra 17: Normed spaces; p-norms.
(A) Linear algebra 18: Norms on R^d are equivalent.
(A) Linear algebra 19: Convex set induces a norm.
(A) Linear algebra 20: Spaces of continuous and differentiable functions.
(A) Linear algebra 21: Lp-spaces of integrable functions.
(A) Linear algebra 22: Scalar product.
(A) Linear algebra 23: Orthogonal vectors and basis.
(A) Linear algebra 24: Orthogonal matrices.
(A) Linear algebra 25: Symmetric matrices.
(A) Linear algebra 26: Spectral theorem for symmetric matrices.
(A) Linear algebra 27: Positive definite matrices.
(A) Linear algebra 28: Variational characterization of eigenvalues.
(A) Linear Algebra 29: Singular value decomposition.
(A) Linear Algebra 30: Rank-k-approximation, matrix norms.
(A) Linear Algebra 31: Pseudo-inverse of a matrix.
(A) Linear Algebra 32: Continuous linear functionals and operator norm.
(A) Linear Algebra 33: Dual space, Riesz representation theorem.
(C) Calculus 1: Sequences and convergence.
(C) Calculus 2: Continuity.
(C) Calculus 3: Sequences of functions; pointwise and uniform convergence.
(C) Calculus 4: Differentiation on R.
(C) Calculus 5: Riemann integral on R.
(C) Calculus 6: Fundamental theorem of calculus on R.
(C) Calculus 7: Power series.
(C) Calculus 8: Taylor series.
(C) Calculus 9: Sigma-Algebra.
(C) Calculus 10: Measure.
(C) Calculus 11: Lebesgue measure on R^n.
(C) Calculus 12: A set that is not Lebesgue-measurable.
(C) Calculus 13: The Lebesgue integral on R^n.
(C) Calculus 14: Differentiation on R^n: partial derivatives.
(C) Calculus 15: Differentiation on R^n: total derivative.
(C) Calculus 16: Differentiation on R^n: directional derivative.
(C) Calculus 17: Differentiation on R^n: Higher order derivatives.
(C) Calculus 18: Minima, maxima, saddlepoints.
(C) Calculus 19: Matrix calculus.
(P) Probability theory 1: Definition of a probability measure.
(P) Probability theory 2: Different types of measures: discrete, with density; Radon-Nikodym.
(P) Probability Theory 3: Different types of measures: singular measures, Lebesgue decomposition.
(P) Probability Theory 4: Cumulative distribution function.
(P) Probability Theory 5: Random variables.
(P) Probability Theory 6: Conditional Probabilities.
(P) Probability Theory 7: Bayes theorem.
(P) Probability Theory 8: Independence.
(P) Probability Theory 9: Expectation (discrete case).
(P) Probability Theory 10: Variance, covariance, correlation (discrete case).
(P) Probability Theory 11: Expectation and covariance (general case).
(P) Probability Theory 12: Markov and Chebyshev inequality.
(P) Probability theory 13: Example distributions: binomial, poisson, multivariate normal.
(P) Probability theory 14: Convergence of random variables.
(P) Probability theory 15: Borel-Cantelli.
(P) Probability theory 16: Law of large numbers, Central limit theorem.
(P) Probability theory 17: Concentration inequalities.
(P) Probability theory 18: Product space and joint distribution.
(P) Probability theory 19: Marginal distribution.
(P) Probability theory 20: Conditional distribution.
(P) Probability theory 21: Conditional expectation.
(S) Statistics 1: Estimation, bias, variance.
(S) Statistics 2: Confidence sets.
(S) Statistics 3: Maximum likelihood estimator.
(S) Statistics 4: Sufficiency, identifiability.
(S) Statistics 5: Hypothesis testing, level and power of a test.
(S) Statistics 6: Likelihood-ratio tests and Neyman-Pearson lemma.
(S) Statistics 7: p-values.
(S) Statistics 8: Multiple testing.
(P) Probability 17a: Glivenko-Cantelli theorem.
(S) Statistics 9: Non-parametric tests (rank and permutation tests, Kolmogorov-Smirnov).
(S) Statistics 10: The bootstrap.
(S) Statistics 11: Bayesian statistics.
(H) High-dimensional spaces.
Statistical Machine Learning Part 7a - What is a convex optimization problem?.
Statistical Machine Learning Part 15 - Convex optimization, Lagrangian, dual problem.