PMR 2025: Optional material

While selected content from the resources below may form part of the compulsory reading, the material is generally not required reading. In particular, you will not need to procure any books for the course.

Further reading

This is a collection of additional material that may help you to better understand the lectures.

Directed graphical models

• Bishop Sections 8.1 and 8.2
• Barber: Chapter 2, Sections 3.1 and 3.3 (without 3.3.6)
• Section 2.1 in Michael Jordan's "An Introduction to Probabilistic Graphical Models" 

Undirected graphical models

• Bishop: Section 8.3
• Barber: Section 4.1, 4.2
• Section 2.2 in Michael Jordan's "An Introduction to Probabilistic Graphical Models" 

Expressive power and comparison between directed and undirected graphical models

• Barber: Section 4.5

Factor graphs

• Bishop: Section 8.4.3
• Barber: Section 4.4

Exact inference

• Bishop: Section 8.4 till 8.4.6 on inference in graphical models
• Barber: Sections 5.1 to 5.4 on variable elimination, sum-product algorithm, etc; Section 23.1 on Markov chains, Section 23.2 on HMM inference

Causality

• Pearl: Chapters 1 and 3

Learning, factor analysis, and ICA

• Murphy's notes on conjugate Bayesian analysis of the Gaussian distribution
• Barber: Sections 9.1 to 9.4 on learning; Chapter 21 on factor and independent component analysis
• More details on PCA (Chapter 2) and background on linear algebra (Appendix A)
• ICA demo

Sampling and Monte Carlo

• Barber: Chapter 27

Variational inference and EM algorithm

• Barber: Chapter 11
• Shakir Mohamed's tutorial on variational inference

Books

• Pattern Recognition and Machine Learning 
  Christopher Bishop 
  Springer 2006
  (available online)
• Bayesian Reasoning and Machine Learning
  David Barber
  Cambridge University Press 2012
  (available online)
• Mathematics for Machine Learning
  Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong
  Cambridge University Press 2020
  (available online)

Advanced materials

These readings typically explore ideas that go beyond what we cover in class. They build on the concepts you have learned and is for those interested in going further.

Papers

• Chapter 4 of Michael Jordan's "An Introduction to Probabilistic Graphical Models" (file is provided as chapter16.ps)
• Research paper "Extending Factor Graphs so as to Unify Directed and Undirected Graphical Models", UAI 2003, by B. Frey
• Research paper "Factor Graphs and the Sum-Product Algorithm", 2001, by F. Kschischang et al.
• Research paper "Causal Diagrams for Empirical Research", 1995 by J. Pearl
• Advanced review on ICA by A. Hyvarinen
• Review paper on further estimation methods for unnormalised models by M. Gutmann and A. Hyvarinen
• Original paper on score matching by A. Hyvarinen
• A general framework to estimate unnormalised models by M. Gutmann and J. Hirayama
• Noise-contrastive estimation to estimate unnormalised models by M. Gutmann and A. Hyvarinen
• Score matching for generative modelling by Y. Song and S. Ermon
• Iain Murray's tutorial on Monte Carlo methods
• An Introduction to MCMC for Machine Learning by Andrieu et al
• Introducing Monte Carlo Methods with R by Robert and Casella
• Pareto Smoothed Importance Sampling by Vehtari et a
• An Introduction to Variational Autoencoders by Kingma and Welling
• Variational Inference: A Review for Statisticians by Blei et al
• Advances in Variational Inference by Zhang et al

Books

• Bayesian networks : an introduction
  Timo Koski and John Noble
  John Wiley 2009
  (available at the University library)
• Graphical models
  Steffen L. Lauritzen
  Oxford University Press 1996
  (available at the University library)
• Probabilistic Graphical Models : Principles and Techniques 
  Daphne Koller and Nir Friedman 
  MIT Press 2009 
  (available at the University library)
• Causality
  Judea Pearl
  Cambridge University PressCambridge University Press, 2009, 2nd ed
  (electronic copy available at the University library)
• An Introduction to Probabilistic Graphical Models (incomplete draft)
  Michael Jordan 
  (parts available online here and here)
• Independent Component Analysis 
  Aapo Hyvarinen, Juha Karhunen, and Erkki Oja 
  (electronic copy available at the University library)
• Monte Carlo Statistical Methods
  Christian Robert and George Casella
  Springer 2004
  (available at the University library)
• Bayesian data analysis
  Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin
  CRC Press 2013
  (available at the University library)
• An Introduction to Sequential Monte Carlo
  Nicolas Chopin and Omiros Papaspiliopoulos
  Springer 2020
  (available at the University library)
• Art Owen
  Monte Carlo theory, methods and examples
  (available online)
• Probabilistic machine learning book series by Kevin Murphy
  (available online)