Week 4: Materials and checklist
Overview of this week
This week we will cover gradient descent, the algorithm used to optimise the weights of neural networks. Last week, we studied how neural language models perform a forward pass, mapping a context into the probability distribution over the next word. Gradient descent is the mechanism for performing a backward pass, from a loss function (the distance between the predicted probability and the empirical probability) to an update of the network weights.
After completing our discussion of language models (generative models), we will introduce a new set of problems around text classification (discriminative models). I will again introduce two families of algorithms to tackle classification problems. One is Naive Bayes, which estimates class probabilities given input texts via counting. The other is a neural classifier, which learns the weights of the function mapping from texts to class probabilities.
Week 3 Materials
- Slides
- Readings
- JM3 7.6-7.7 (*)
- JM3 4.0-4.3 (*), 4.4-4.6, 4.7 (*)
- JM3 7.4 (*)
Tutorial
We also finally have our first tutorial group discussion meetings this week! Tutorials will give you the opportunity to practice the kinds of questions you will encounter in the final exam. Make sure to read the questions in advance of the meeting (expected time 15 minutes) to make the best of the tutorial.
You can find the tutorial questions here [pdf]. The solutions will be released next week.
Our tutors are looking forward to meeting you!
Extra help hours
Also, a reminder that we have extra help hours this week to help with questions about the assignment. The help hour is scheduled for 4 p.m. on Friday 13th October in room G.03 of the Informatics Forum.
Week 4 checklist
- Throughout: continue working on the assignment, which is due next week.
- Before your tutorial group meeting: Prepare for the meeting. If you started reading last week, it should only take you around 15 minutes to prepare for your group, so don't spend ages. But you will need to do the reading and consider the questions in advance!
- Wed-Fri: Participate in your tutorial group discussion (check your timetable for the time).