Week 6 – Initialisation, Normalisation, Pretraining
https://opencourse.inf.ed.ac.uk/MLP
Week 6
Teaching week commencing: 26 October 2026
Key topics: Initialisation, Normalisation, Pretraining.
Reading guide
- Goodfellow, Bengio and Courville: Deep Learning — Section 8.4, Section 8.7.1.
- For keen readers:
- Goodfellow, Bengio and Courville: Deep Learning — Chapter 14.
- References:
- Glorot and Bengio, Understanding the difficulty of training deep feedforward networks, AISTATS-2010
- Ioffe and Szegedy, Batch normalization, ICML-2015
- Kratzert, Understanding the backward pass through Batch Normalization Layer
Lecture materials
Scripted recording: Initialisation, Normalisation, Pretraining
This recording was produced in a previous year. The corresponding lecture content has since changed significantly, so the recording should be treated as supplementary material. References to course logistics are out of date.
Slides: Lecture 06 — Initialisation, Normalisation, Pretraining (see below).
Practical materials
The Lab 5 branch also includes updated Lab 4 files containing the solutions.
Indicative workload
The original taught-course estimate was 7 hours:
- Lab: 1 hour.
- Recorded lecture: 1 hour.
- Lecture/discussion: 1 hour.
- Independent study: 3 hours.
- Additional work on the PyTorch lab component: 1 hour.
Independent study includes further work on the lab and the suggested reading. The lab exercises also help test your understanding of the material.