Week 7

https://opencourse.inf.ed.ac.uk/MLP

Teaching week commencing: 02-Nov-2026

Key topics: Convolutional Neural Networks (CNNs)

Reading guide

Goodfellow et al, chapter 9

Michael Nielsen, Neural Networks and Deep Learning (ch 6)

Yann LeCun et al, “Gradient-Based Learning Applied to Document Recognition”, Proc IEEE, 1998.

Optional Reading on CNNs:

Huang et al, “Densely Connected Convolutional Networks”, CVPR 2017.
He, Kaiming, et al. "Deep residual learning for image recognition." CVPR 2016
Szegedy et al., “Going deeper with convolutions”, CVPR 2015
Simonyan and Zisserman, “Very Deep Convolutional Networks for Large-Scale Visual Recognition”, ICLR 2015
Krizhevsky, Sutskever and Hinton, “ImageNet Classification with Deep Convolutional Neural Networks”, NIPS 2012
Hawkins, Douglas M. "The problem of overfitting." Journal of chemical information and computer sciences 44.1 (2004): 1-12
Dayhoff, Judith E., and James M. DeLeo. "Artificial neural networks: opening the black box." Cancer: Interdisciplinary International Journal of the American Cancer Society 91.S8 (2001): 1615-1635.

Lecture materials

Scripted recording: Convolutional Neural Networks (1), Convolutional Neural Networks (2)

This recording was produced in a previous year. The corresponding lecture content has since changed significantly, so the recording should be treated as supplementary material. References to course logistics are out of date.

Slides: MLP 26-26 Convolutional Neural Networks + MLP 25-26 Handcrafting a CNN + [Add-on] Comparing a CNN and FNN & Logic Operations (see below).

Practical materials

Labs: Lab6 (the branch for Lab6 also updates the files for Lab5 so that they include the solutions)

Milestones: Getting to craft some Neural Nets in-class :)

Expected Effort Estimate: 7 hours

Attendance/Timed Events
Lab - 1 hour
Recordings - 1 hour
Discussion/Lecture - 1 hour
Study - 3 hours ( +1 hour for PyTorch Lab component)

"Study" can include extra time working on labs, suggested reading, and Piazza. The Labs (from Week 2 and on) are generally a good starting point as they also test your understanding of the material.

(Semester 1 Total Expected Effort <= 82 hours = 67 hours studying + max 15 hours coursework) [The standard guidelines are 10 credits = 100 hours, so we plan for less to allow for a range of needs].

Extra [Thinking like an Engineer] - What is Artificial Intelligence?

Slides: What is Artificial Intelligence? (see below)

In studying Machine Learning (ML) and Artificial Intelligence (AI), it’s easy to get lost in the details—the theory, mathematics, and programming. To help you see the forest instead of just the trees, let’s simplify AI to its core idea:

AI is fundamentally about automating information processing.

Think of it this way:

Specified information comes in—this input must be consistently defined.
Processing happens—imagine this as a “black box” that transforms the input.
Specified information comes out—again, consistently defined.

Often, this processing step might repeat or feed into another process (like in Reinforcement Learning). By adopting this simple, structured view of ML/AI, you can better analyse both the literature and the problems you encounter, focusing on what processing is taking place or needed.

A problem might call for a single ML task, like an end-to-end solution (e.g., image classification), or it might require breaking down the problem into a system involving several ML tasks and other processes. For instance, imagine designing a chat-bot that also runs a separate AI profanity filter or a control system that runs a different process depending on a classifier over user input.

Once you identify these sub-tasks, you can start proposing and critiquing solutions. Each sub-process could influence the design requirements of other parts of your system. By understanding the parts, it’s easier to question assumptions and refine design choices.

Let’s put this thinking into practice.

Consider the following:

When faced with a new problem, is your first instinct to Google a paper that solves the exact same problem?
That’s useful, but start instead by breaking down your problem. Identify its components, and think of similar problems you know of.
Then look for solutions to these smaller parts. Can you build a solution from pieces that already exist?

Practice Scenario (hypothetical, remember real-life ethical and legal considerations apply):

Imagine you live in a shared space, and someone has been stealing milk. You have access to a camera feed pointed at the fridge. You want an alarm to sound when someone takes milk that doesn’t belong to them.

Challenge: Design an AI system to solve this.

This problem is certainly different from what you might easily encounter in the published literature, but you can break it down into sub-processes. Identify these parts and find existing technology to handle each one. What approach could work?