Week 7: Transformers (cont.) and exam prep

Reminders and announcements

Welcome to Week 7! As of Monday this week, we will have finished all the content necessary for the assignment, so we can slow down now in terms of new material. This should give you some time to consolidate what you have learned and to work on the assignment.

• Lab 4 is this week. This lab will lead you through implementing and interpreting multi-head attention, which is directly relevant to the assignment. It is recommended to revise the material on Transformers prior to attending.
• Assignment: 
  • Partners will be released by Tuesday.
  • The specification (including helper code and submission templates) will be released by the end of Thursday 5th of November.
• Tutorial exercises for the Week 8 groups are here. The tutorial includes questions on ethical and social issues in NLP. In this tutorial, you will also practice discussing and critiquing research papers.

Overview of this week

This week, we finish the discussion of Transformer architectures, focusing on their input (token and positional embeddings) and outputs (language modelling head). In the second lecture, I will spend some time talking about the exam and suggestions for how to prepare for it. You will notice I am not waiting until the end of the class to do this! That's because doing exam preparation well is not a last-minute operation.

On Friday, we'll have a day off from lecture. I would encourage you to use this time to catch up and/or start revising.

Lectures and reading