Many tasks in NLP involve text generation (e.g., machine translation, text summarization, chat agents). This week, we will consider neural architectures suitable for generation tasks, as well as discuss how to perform generation (decoding strategies) and evaluate the quality of text generation. We will also discuss some specific applications such as neural machine translation. The neural architectures and modeling components (e.g., various ways of modeling attention) will be important for transfer learning (next week) and even for problems that do not involve generation.
The content in this folder is structured as follows:
25: Sequence-to-Sequence Modeling
26: Neural Attention
27: Transformers
As always, each of the above includes videos, the slides that were used in the videos, required readings, and a post-lecture quiz. The quiz is a chance for you to gauge your understanding of the material presented here, and so we strongly encourage you to review this content in the above order, and then complete the quiz. If there is anything you don't understand, then you have several options:
- Post a question on piazza;
- Ask a question at the in person lectures; and/or
- Ask your tutor.