Welcome to Machine Learning Theory
Lectures
Tuesdays 16:10 -- 18:00.
40 George Square, Lecture Theatre LG 09.
Learning Outcomes
On successful completion of this course, you should be able to:
- interpret and explain rigorous statements about properties of machine learning methods.
- evaluate properties of learning models through proofs and examples.
- relate, compare, and contrast the implications of various qualities of machine learning models covered in the course.
- formulate precise mathematical requirements corresponding to desired properties in real learning problems, and explain their decisions.
Course Outline
The following is an indicative list of topics in the course:
- Notations, terminology and formal models.
- Learning theory: Empirical risk minimisation and sampling complexity. Probably approximately correct (PAC) guarantees.
- Complexity of learning models (e.g. VC dimension) and bias-complexity tradeoff.
- Optimization algorithms. Regression, SVM, Stochastic gradient descent and its variants.
- Regularization, convexity, stability, Lipschitzness and other properties
- Statistical Privacy
- Mechanisms for privacy preserving machine learning. Differentially private stochastic gradient descent.
- Interpretable machine learning. (E.g. Feature importance)
- Fairness.
The topics will be discussed with reference to standard machine learning techniques, and examples of realistic problems. Our approach will include precise definitions and analysis as well as examples and intuitive explanations. The relevance and domain of applicability of the various concepts will be discussed.
Tutorials and problem sets will be available to help understanding and exploration of the subject.
License
All rights reserved The University of Edinburgh