Week 4 – Deep Neural Networks (2), ReLU layers, Generalisation and Regularisation.

https://opencourse.inf.ed.ac.uk/MLP

Week 4

Teaching week commencing: 12 October 2026
Key topics: Deep Neural Networks (2), ReLU layers; Generalisation and Regularisation.

Reading guide

• Goodfellow, Bengio and Courville: Deep Learning — Sections 5.2–5.3, Section 6.5, and Sections 7.1–7.5.

Lecture materials

Scripted recording: Deep Neural Networks (2)

This recording was produced in a previous year. The corresponding lecture content has since changed significantly, so the recording should be treated as supplementary material. References to course logistics are out of date.

Slides: Lecture 04 — Deep Neural Networks (2) ReLU layers, Generalisation and Regularisation (see below).

Practical materials

Lab 3 — code and notebooks

The Lab 3 branch also includes updated Lab 2 files containing the solutions.

Indicative workload

The original taught-course estimate was 7 hours:

• Lab: 1 hour.
• Recorded lecture: 1 hour.
• Lecture/discussion: 1 hour.
• Independent study: 3 hours.
• Additional work on the PyTorch lab component: 1 hour.

Independent study includes further work on the lab and the suggested reading. The lab exercises also help test your understanding of the material.