Week 8: Neural networks

Reminders and announcements

• Solutions to Lab 3 are now available.
• Tutorial 3 group meetings are this week. There is a short paper to read to prepare, and an exercise on CKY, so remember to set aside time for these before your group meeting.
• Assignment 2 was released on Monday; please see the bottom of this page for all materials and help hour information.

Overview of the Week

This week we will focus on neural architectures for language modelling. This family of models provides several advantages over n-gram language models, as they can capture semantic relationships via dense word representations and consider unbounded contexts. I will start by presenting the "forward mode" (or inference): given a context and neural weights, what is the probability of the next word? In the second part of the lectures, I will focus instead on the "backward mode" (or learning): given the model's prediction and its loss, how should its weights be updated? Over the course of the week, I will introduce different neural architectures, ranging from log-linear LMs and multi-layer Perceptrons (MLPs) to Recurrent Neural Networks (RNNs).

Lectures and reading

Assignment 2 information

On Monday afternoon/eve, we will post here:

• The assignment handout. The specifications for the assignment, including links to the code and data you will use.
• Submission templates. [.tex] [.docx] You will need to use one of these files to prepare your answers, then compile it to a .pdf for submission.
• Partner assignments. (Requires UoE login.) Please contact your partner (if any) as soon as possible to start working together on the assignment. As mentioned last week, we only partnered up people who asked us to. If you find any problems with the partnering, please contact the TA Mai Dao immediately.

Help hours for assignment 2

We again have some extra help hours for the assignment. All help hours are in AT 5.07.