NLU-11: Natural Language Understanding, Generation, and Machine Translation

[Quick Navigation during the semester: Course materials / Tutorials / Labs / Assignments / Time table ]

Welcome

Hello, we are pleased to welcome you to the Natural Language Understanding, Generation, and Machine Translation course.

This is an advanced course on natural language processing, focusing on deep learning methods. Please have a look at the Enrollment Guidance on the Course Information page -- it contains important information you should know before enrolling for this course. Please note in particular that passing ANLP (or equivalent) is a prerequisite for taking this course. This will not normally be waived.

Course Information at a Glance

This course consists of three lectures per week and one tutorial or lab session per week (these alternate and start in week 3).
Each week we release a webpage in the Course Materials page. This contains everything you need for that week: the lecture slides, the readings, the tutorials or the lab sessions and the assessments. These will be released at 7pm the Friday before the week begins - please check it before the week starts.
The Tutorials and Labs pages gives an overview of all the tutorials and labs and all the materials provided so far.
Please check the course time table. This contains the days/times for the lectures and for all tutorials and lab sessions. You should attend one tutorial or lab per week (they alternate weekly).
Lectures will be in person. If you can't attend in person, please watch the lectures on the Live Classroom page, either via live stream or recordings.
If you're enrolled for this course, you will be automatically assigned a tutorial group in week 1. If you are unhappy with your group, please use the group change request form. If you joined the course late and have not yet been assigned a tutorial, please contact the ITO. Labs are dropin.
Tutorials and labs are also in person.
Please use the EdStem discussion forum for any questions about the course content.
The TAs will hold weekly office hours when the assignments are out, starting in week 2.
The coursework for this course will be accessible from the Assessment folder on Learn.
The course lecturers are Alexandra Birch and Shay Cohen with guest lectures by Frank Keller, Emily Allaway and Nikolai Malkin. The course organiser is Alexandra Birch. The teaching assistants (TAs) are Leonardo Ranaldi and Gustavo Cilleruelo. Please see this contact page for contact details.
For any administrative issues (e.g., if you want to change tutorial), please use the ITO contact form. Information on late submission of courseworks and extension requests can be found here.

Enrollment Guidance: Who should take this course?

NLU+ is not an introductory course. It is the second course of a two-semester sequence in natural language processing, and assumes that you have substantial prior knowledge of three areas:

Natural language processing
Machine learning
Programming

"Substantial" means the equivalent of a foundational course. PATH gives guidance on specific prerequisite courses, though the statement is a bit confusing. Below we give more detail.

For natural language processing, there are three alternative ways to demonstrate the prerequisite knowledge. Any one of these suffices:

Accelerated Natural Language Processing (ANLP). If you are an MSc student or an undergraduate who spent third year abroad, you should take this course before taking NLU+.
Foundations of Natural Language Processing (FNLP). If you are a BSc or MInf student, you should take this sequence before NLU+. Note that FNLP is not a co-requisite to NLU+. That is, you must take it before you take NLU+. If you take the courses concurrently, then you will encounter many concepts in NLU+ before you see the relevant foundational material in FNLP, and you'll fall behind easily. So, if you are in your third year, you should take FNLP, and then you can take NLU+ in your fourth (or fifth) year.
Another course equivalent to one of the above sequences. If you have taken NLP elsewhere, compare your syllabus to the ANLP syllabus. Are you comfortable with 80% or more of the topics? Do you have enough time in your already busy schedule to self-study the rest? If yes, then you should be ok. If no, then you should take one of the above paths before you take NLU+. Due to volume of requests, the instructors are not able to evaluate your syllabus for you.

For machine learning: any one of Introductory Applied Machine Learning (IAML), Machine Learning and Pattern Recognition (MLPR), Machine Learning Practical (MLP), or equivalent suffices. It is ok to take MLP concurrently with NLU+, since it is a year-long course. It does not matter which of these courses that you take, provided you have taken one (or its equivalent). Students on the Speech and Language Processing MSc, for example, usually revise IAML, and have generally done very well on the NLU+ material.

For programming: Programming skills at least at the level of Computer Programming for Speech and Language Processing.

Self-assessment of preparedness

If you are not certain that you are prepared for the course, you are advised to do the following during the first two weeks of the semester.

Review the 2023 or 2024 ANLP exam. Are you comfortable with the material?
Look over the NLU+ syllabus. Does this look like material that you are prepared to learn?
Carefully read the first coursework when it is released in week 2. Is it work that you are prepared to do?

If your answers to these questions are positive, then you can be confident that you are prepared for NLU+. If you answer no to any of these questions, then you may want to take another course.

Many students will have taken exams for ANLP and/or machine learning courses in December. If the marks for those courses are available in time for semester 2 registration, then use them: if you receive a failing or borderline mark in your NLP, ML, or programming prerequisite course, then you should not take NLU+. If marks are not available, then high coursework marks in those courses are an indicator of preparedness (i.e. above 65; this is because coursework marks are typically much higher than exam marks). But if you are genuinely concerned and your attendance in NLU+ is contingent on exam results for one of these courses, then you are advised to follow lectures from another semester 2 course, and talk to your personal tutor immediately about switching to that course if you receive negative exam results later in the semester. (This option should be possible in exceptional circumstances, and failing a prerequisite course is certainly exceptional).

NLU+ is not difficult as long as you are adequately prepared, and the guidance is not intended to discourage you from taking the course. It is intended to help you determine whether you are adequately prepared. If you aren't, we advise you to first take the missing prerequisite courses so that you have a good experience when you take NLU+ (or a similar course) in the future.

How to Get Help on Course Material

EdStem discussion form

If you are still confused about something after reviewing the material in learn, feel free to ask questions in EdStem. You can ask and answer questions anonymously (to your classmates) or privately to the instructors. Please do not ask us questions over email. The reasons we ask that you ask in EdStem rather than over email:

We get a lot of email, and it's very easy for your message to get lost. However, we will read EdStem at least once a day.
Many students have the same question. When we answer questions in EdStem, everyone benefits.

Even though we will do our best, the course staff is still very small, and the number of students is very large. Waiting for instructors to answer all questions creates a communications bottleneck, and ignores the largest group of people in the class: students. So what will be most effective for your learning is if you also answer questions in EdStem. Answering another student's question doesn't just benefit them, it benefits you: if you can answer a question, you know that material. We will endorse good answers, and clarify any answers that are unclear or incorrect. Please use this as a learning experience.

The other reason for students to answer questions is that you will receive answers more quickly. Instructors will review and answer questions once a day, Monday through Friday. Do not expect immediate answers from the instructors.

TA office hours for general support on course material

It really helps to discuss your understanding of the material with other people, and we encourage you to form study groups. But if you want to talk to someone with more experience, please drop in to the weekly TA office hours, which start in week 2.

Labs for coursework support

The labs happen in weeks 3, 5, 7, and 9. The lab exercises are designed to practice material that you will need for the assignments. At each lab, there will be two demonstrators at hand, who will be able to help with the lab exercises and answer questions relating the assignments.

Tutorials for content support

The tutorials happen in weeks 4, 6, 8, and 10. The tutorial exercises are designed to practice material that you will need for the exams. Before each tutorial you will attempt to answer the questions and during the tutorial you will discuss the answers with your tutor.

License