Week 3 – Deep Neural Networks (1), Hidden Layers, Activation functions
https://opencourse.inf.ed.ac.uk/MLP
Week 3
Teaching week commencing: 5 October 2026
Key topics: Deep neural networks, hidden layers and activation functions.
Reading guide
- Goodfellow, Bengio and Courville: Deep Learning — Sections 6.1, 6.3 and 6.4.
Further reading
- On the Number of Linear Regions of Deep Neural Networks — optional reading for those wishing to explore the topic further.
Lecture materials
Scripted recording: Deep Neural Networks (1)
This recording was produced in a previous year. The corresponding lecture content has since changed significantly, so the recording should be treated as supplementary material. References to course logistics are out of date.
Slides: Lecture 03 — Deep Neural Networks (1), Hidden Layers, Activation Functions (see below).
Practical materials
The Lab 2 branch also includes updated Lab 1 files containing the solutions.
Indicative workload
The original taught-course estimate was 7 hours:
- Lab: 1 hour.
- Recorded lecture: 1 hour.
- Lecture/discussion: 1 hour.
- Independent study: 3 hours.
- Additional work on the PyTorch lab component: 1 hour.
Independent study includes further work on the lab and the suggested reading. The lab exercises also help test your understanding of the material.