NLU-11: Week 2 | Open Course Materials

Assessment 1

This will be released first thing this Monday 22 on the Assessment Page in Learn and the deadline for handing it in is 12pm Friday 16 February. The coursework will be done in pairs, please find a partner and submit the Partner Form by 5pm on Tuesday 23 January.

Labs 1: Numpy Lab

Labs will start next week. They will be made available on the Labs page at the beginning of this week. These are drop-in sessions where you can work through exercises that will help with the assignments.

Slides	Lecture	Course Content
Feedforward Language Models	4 (Birch)	Required reading: Section 5 of Neural Machine Translation and Sequence-to-sequence Models: A Tutorial, Neubig. Background reading: Chapter 7 of Speech and Language Processing, 3rd edition, Jurafsky & Martin. Recommended revision: Logistic regression: Chapter 5 of Speech and Language Processing, 3rd edition, Jurafsky & Martin
Recurrent Neural Networks	5 (Birch)	Required reading: Sections 6 of Neural Machine Translation and Sequence-to-sequence Models: A Tutorial, Neubig. Backpropagation through Time, Jiang Guo. Background reading: RNNs & LSTMs: Chapter 9 of Speech and Language Processing, 3rd edition, Jurafsky and Martin Understanding LSTM networks blogpost (Olah 2015) Supervised Sequence Labelling with Recurrent Neural Networks, (Graves 2012) Long short-term memory, (Hochreiter and Schmidhuber 1997) On the Properties of Neural Machine Translation: Encoder–Decoder Approaches. (GRUs Cho 2014)
Modelling data and words	6 (Birch)	Required reading: Neural Machine Translation of Rare Words with Subword Units (BPE Sennrich et al. 2016) Additional reading: BPE-Dropout: Simple and Effective Subword Regularization, Provilkov et al. (2020)

License