Accelerated Natural Language Processing

ANLP: What to do if you have questions

flittlet — Tue, 18 Jul 2023 11:35:37 +0000

ANLP: What to do if you have questions flittlet Tue, 18/07/2023 - 12:35

Questions about course content

Here are your options:

Other students: We try to structure the class so that you will meet other students early on. We encourage you to discuss course material and questions amongst yourselves, provided you are not sharing solutions to assessed work (homework assignments).
Piazza discussion forum: If you still have questions after talking to your classmates, please use the Piazza discussion forum. This is typically the quickest way to get an answer to your question because other students may be able to answer right away, and course staff will try to answer within a day or two if not. Your post can be anonymous to other students. You can also post a private question (visible to instructors only) if your question is of a personal nature or would give away solutions to assessed work.
Help hours with TAs, or lab demonstrators:
- We also have at least one help hour per week with a TA. This is a drop-in session where you can go if you have questions but it is completely optional. It may be useful if you have more detailed questions than what we can answer in the session with lecturers, or if you are still confused after that. In some weeks we will schedule additional TA hours to help with assignment-related questions.
- Lab demonstrators are available during the scheduled lab sessions to answer questions about lab material.

Questions about course administration

For questions about course administration, such as registration issues, please contact Informatics Student Services (ISS) using the form linked under Course Contacts below.

Course Registration and Change (for taught MSc students)

Questions about extensions or issues affecting your work

For extension requests, or to request special circumstances, you need to use the University's central Extensions and Special Circumstances system. It is recommended that you first contact your Student Advisor or the Student Support team in your School.

Questions about technical or computer problems

For help on access, email, wireless, VPN, teaching tools (such as Learn, Gradescope, and Piazza), and Microsoft tools (such as Teams and Office 365), contact the central University’s IT support helpline. For issues related to the Informatics local computing environment (DICE), including DICE accounts and remote access to DICE, contact the Informatics Computing Helpdesk.

Questions for individual course staff

Please email individual course staff if and only if:

your question is of a personal nature, and
your question can only be handled by the specific person you are contacting.

Otherwise, please post to Piazza (using a private message if appropriate) or use one of the other methods described above.

Questions about anything not covered on this page

Please consult Course Information > Help and Support on the course's learn page.

License

ANLP: Weeks 11-12: Deadlines and exam revision

flittlet — Tue, 18 Jul 2023 11:35:33 +0000

ANLP: Weeks 11-12: Deadlines and exam revision flittlet Tue, 18/07/2023 - 12:35

Information for Week 11 and beyond

Welcome and overview of the week

Hello everyone and welcome to the final wrapup and revision weeks for the course!

As you know, we don't have any new content from now on, but I thought I'd use this place for some reminders and announcements.

How to get help while revising for the exam

Piazza: We'll continue monitoring Piazza in the run-up to the exam, but:
- We will not answer questions posted less than 24 hours before the exam. This is to encourage you to start revision early, to discourage a flood of last-minute questions, and so we can catch up on anything posted earlier. Of course you're still welcome to post, in case your classmates can answer.
Additional help hours.
- Friday, 1 December 16:00-17:00 (Location: Informatics Forum, room 4.02)
  
  For all remaining help hours, the Location is 21 George Square, room G.02
- Monday, 4 December 11:00-12:00
- Tuesday, 5 December 11:00-12:00
- Wednesday, 6 December 10:00-11:00
- Thursday, 7 December 12:00-13:00
- Monday, 11 December 14:00-15:00
- Tuesday, 12 December 16:00-17:00
- Wednesday, 13 December 10:00-11:00
- Thursday, 14 December 15:00-16:00

Reminders about the exam

You can find the exam revision guide under 'Assessment' in learn.

Good luck!

Finally, good luck with your assignment and exam revision! We hope you've enjoyed the course!

License

ANLP: Readings

flittlet — Tue, 18 Jul 2023 11:35:33 +0000

ANLP: Readings flittlet Tue, 18/07/2023 - 12:35

Main readings

The main readings for ANLP derive from two editions of the Jurafsky and Martin textbook:

Second edition (denoted JM2)
Third edition (denoted JM3)

These can be found on the ANLP Resource list. In order to view some resources on the list, you may need to be logged in with your EASE account or DICE account.

Accelerated Natural Language Processing Resource List

For more information on getting the most out of your courses Resource List, have a look at this video.

Optional readings

Linguistics background

In previous years some Informatics students have asked for more background reading on linguistics. A good place to start might be this text, which is also on the resource list, and available online through the University library:

Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax by Emily M Bender. Synthesis Lectures on Human Language Technologies, June 2013, Vol. 6, No. 3 , Pages 1-184.

Further mathematical details

Some students may want a more rigourous treatment of the models and machine learning methods we discuss. In that case I suggest the following textbook. It covers many of the same topics we do, but assumes somewhat more background and comfort with formal methods.

Introduction to Natural Language Processing by Jacob Eisenstein. MIT Press, 2019. (Draft version is available for free from author's github page here.)

Weekly optional readings

Other optional readings related to each week's topics are provided below for students who wish to learn more details, especially about recent research in the area. Some of these papers may also give you ideas for your IRR review. Many of the optional readings assume additional mathematical or machine learning background beyond what is covered in this course, but you may be able to understand the general idea of these papers by reading the introduction and skimming the rest, even if you cannot understand all of the details.

In the schedule below, we use the following key to optional readings:

(A): These readings provide more detail about the week's topics, without requiring much additional knowledge of machine learning or later parts of this course.
(B): The main concepts required to understand these readings are covered by the end of this course (though perhaps not all details).
(M): These readings assume significant mathematical or machine learning background beyond what is covered in this course.

Week 1

(A) Liu et al. (2023). We’re Afraid Language Models Aren’t Modeling Ambiguity. Evaluates modern large language models (LLM) on their ability to recognise and disentangle ambiguity, finding that they often fail to do so.
(B) Schone and Jurafsky (2001). Knowledge-Free Induction of Inflectional Morphologies. Uses relatively simple methods to combine multiple sources of information, aiming to learn morphological relationships from a corpus without annotation.
(B) Kirov et al. (2017). A Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax. Uses dependency syntax and neural models, which we will discuss later in this course.
(B) Faruqui et al. (2016). Morpho-syntactic Lexicon Generation Using Graph-based Semi-supervised Learning Uses a graph-based method to predict morpho-syntactic information for large lexicons from small seed lexicons, for a variety of languages.

License

ANLP: Week 9 Lab solutions

flittlet — Tue, 18 Jul 2023 11:35:32 +0000

ANLP: Week 9 Lab solutions flittlet Tue, 18/07/2023 - 12:35

Week 9 lab solutions [pdf]

License

ANLP: Week 9 self-assessment solutions

flittlet — Tue, 18 Jul 2023 11:35:32 +0000

ANLP: Week 9 self-assessment solutions flittlet Tue, 18/07/2023 - 12:35

Self-assessment solutions [pdf]

License

ANLP: Week 10: Ethics and Bias + guest lectures

flittlet — Tue, 18 Jul 2023 11:35:32 +0000

ANLP: Week 10: Ethics and Bias + guest lectures flittlet Tue, 18/07/2023 - 12:35

Week 10: Welcome and checklist

Welcome and overview of the week

Welcome to Week 10, the final week of the course! It simultaneously feels like a very long time and just yesterday that we started this course together. I hope you will all take a moment to congratulate yourselves on making it this far, and to reflect on what you have learned so far.

This week is focused on some bigger picture problems that we need to think about when building NLP systems: ethics and bias. We will have two guest lectures: one on the main topic, and another on semantic parsing. Both guest lectures are from researchers currently working in these areas, and will give you a glimpse of the current state of thinking in the field. They will also give you a small taste of NLU+, which most students take after ANLP.

Lecture Materials

Lecture 1 [pdf] Readings: Hovy and Spruit 2016 (*)
Lecture 2 [slides]: Guest lecture by Seraphina Goldfarb-Tarrant and Eddie Ungless on bias in NLP
Lecture 3: Guest lecture by Laura Perez-Beltrachini on semantic parsing

Additional Materials

The week 10 tutorial [pdf] will give you practice answering real exam questions, from a previous year's exam. We recommend that you treat this as a practice exam: set a timer for 45 minutes, and write your answers on paper. The questions span material from the whole semester, and will test your knowledge of multiple topics. [solutions]

Exam revision guide

An exam revision guide is now available under the "Assessment" tab in learn. We do not plan to hold any extra revision lectures during Revision Week, because we have already provided a lot of exam preparation throughout the semester (e.g., discussing questions from past papers, the practice test, etc).

Week 10 checklist

By Wednesday at noon: submit your solutions to assignment 2!
On Wed/Thu/Fri: attend your tutorial group. You should work through the tutorial sheet first, ideally treating it as exam practice.
Before the start of next week: Work through all course self-assements and quizzes.
- If you haven't been doing the self-assessments and quizzes, we strongly recommend that you do so. You should attempt to answer each question on your own before looking at the solution, because you need to practice doing the work. If you just read at the question and then read the solution, without first attempting to answer it on your own, you can easily convince yourself that you know the answers. But this is not good practice for the exam, where you need to produce the answers!
Next Tue (Week 11):
- There will be no more lectures.
- Do something nice to celebrate!
- And then start revising for the exam.

License

ANLP: Week 9: semantics beyond the word level

flittlet — Tue, 18 Jul 2023 11:35:31 +0000

ANLP: Week 9: semantics beyond the word level flittlet Tue, 18/07/2023 - 12:35

Week 9: Welcome and checklist

Welcome and overview of the week

Welcome to Week 9! As we near the end of the semester, it's always a difficult time of year. I expect that many of you (like us) are feeling the strain of working hard for so long, you probably have several coursework deadlines coming up, and of course the weather is colder and darker as well.

So, you will probably be glad to hear that the technical content will be one of the lighter ones this week. On the technical side, we'll talk more about semantics, but now at the level of sentences rather than words. In the first two levels, we'll cover some classic approaches to semantices, and in the third lecture, we'll take a tour of many additional problems that arise in semantics.

Lecture Materials

Lecture 1 [pdf] Readings: JM3 19.0-19.3.2(*), 19.3.4 (*),
Lecture 2 [pdf] Readings: JM3 19.3.3(*), 19.3.5-19.4.0 (*), JM2 18.0-18.3.0 (*)
Lecture 3 [pdf] Optional: Bender and Lascarides 2019 (many examples in lecture are from this source)

Additional Materials

The week 9 lab [pdf] will give you some hands-on practice with classification algorithms. It uses the same library (scikit-learn) as the assignment, so completing the lab will help you understand the assignment.
A self-assessment sheet [pdf] will help you practice exercises in semantics. The solutions will become available at the bottom of this page next week.
Quizzes will become available in gradescope after each lecture, for further practice.
Week 8 tutorial solutions [pdf].

How to get help this week

We also have various options for you to get help this week, especially about assignment 2:

For technical questions about the assignment (e.g. help with understanding and using the code):
- We will have four additional help hours:
  - 15:00 Monday 13 Nov, 24 Buccleuch Place room 1.10
  - 15:00 Tuesday 14 Nov, 24 Buccleuch Place room 1.06
  - 16:00 Thursday 16 Nov, 24 Buccleuch Place room 1.06
  - 11:00 Friday 17 Nov, 24 Buccleuch Place room 1.08
- You can also post a private question on Piazza. We will make the question and answer public if that is appropriate.
For conceptual questions about the assignment, please post a private question on Piazza. We will make the question and answer public if that is appropriate.
For non-assignment related questions, we have the usual online help hour on Wednesday morning at 10am in 19 George Square, room 1.10.

Week 9 checklist

Throughout the week: continue working on the assignment, which is due on 22nd November at noon.
Mon/Tue afternoon: attend your lab session.
Before the start of next week: Work through all quizzes and self-assessment exercises.

License

ANLP: Week 8: Lexical semantics and word embeddings

flittlet — Tue, 18 Jul 2023 11:35:30 +0000

ANLP: Week 8: Lexical semantics and word embeddings flittlet Tue, 18/07/2023 - 12:35

Week 8: Welcome and checklist

Content this Week

This week we will focus on lexical semantics – the problem of representing words, whether in a “sparse” way (such as long vectors with many zeros) or in a “dense” way (such as word embeddings). We will also discuss a useful resource in NLP – WordNet, which categorises words and relations between them. You can find the slides here:

Lecture 1 [pdf] Reading: JM3 6.0-1 (*), 23.0-2 (*). 23.3-4.
Lecture 2 [pdf] Reading: JM3 6.2-7 (*)
Lecture 3 [pdf] Reading: JM3 6.8 (*), 6.10 (*)

Discussion / Tutorial 3 Questions

This week's discussion sessions will be based on questions from previous years' exams. There are three questions, one about parsing with CKY, one about estimating a PCFG and one about transition-based parsing. You can find the questions here: [pdf]

Assignment 2: tasks

The assignment has been released. As before, we will also schedule extra help hours for the assignment. More details will be given next week.

Assignment 2 (slot labeling):
- The tasks for assignment 2 are now available [pdf].
- The code and data are also available [zip].
- You can find your group for the assignment here [txt] (students working alone have not been listed). Please contact your partner (if any) as soon as possible to start working together on the assignment. To do so, you can simply email UUN@sms.ed.ac.uk (your partner's UUN is listed in both the .txt and .cvs files).
- Write your answers on the submission template [tex] [docx], compile a pdf and upload it on Gradescope by November 22nd at noon.

Week 8 checklist

By Tuesday at 17:00: complete the coursework 2 pairing form.
As soon as possible after partners are allocated: contact your assignment partner so you can prepare for the assignment, especially if you want to work on the lab together in preparation.
Complete this week's quizzes.
Attend the discussion group.
From Thursday: carefully review Assignment 2.
By the end of the week – work through all lectures.

Enjoy your week of learning!

License

ANLP: Week 7 Lab solutions

flittlet — Tue, 18 Jul 2023 11:35:30 +0000

ANLP: Week 7 Lab solutions flittlet Tue, 18/07/2023 - 12:35

Solutions for the lab [html] [pdf]

License

ANLP: Week 7 Tutorial solutions

flittlet — Tue, 18 Jul 2023 11:35:29 +0000

ANLP: Week 7 Tutorial solutions flittlet Tue, 18/07/2023 - 12:35

Week 7 tutorial solutions [html] [pdf]

License