Week 3: Classification and lexical semantics

Reminders and Announcements

Welcome to the start of our third week! We are seeing some good questions on Piazza (and thanks especially to those of you who have answered other students' questions!), and we hope you all enjoyed the first lab! Our demonstrators thought it went well, but if you do have any suggestions on how to improve it for next year, please let us know since this is the first time we've run that lab.

I can see lots of you are getting to know each other and talking after lecture, which is good to see. If you are new to Edinburgh, I hope you're also getting to know the city a little bit. Here are some reminders for this week:

• We have our second lab this week, and then we'll switch to tutorial groups for Weeks 4 and 5.
• Tutorial groups do require preparation -- see under Additional Materials below.
• Important: If you are in either of the Tuesday tutorial groups (Tue 14:10 or Tue 16:10, starting on 7 Oct), the timetable currently shows these groups meeting in weeks 4, 6, 8, and 11. However, the week 6 meeting is incorrect, and will be happening in week 5! (All tutorial groups meet in weeks 4, 5, 8, and 11.) We have asked the timetabling service repeatedly to fix this, and will keep doing so, but in the meantime please update your own diary to indicate that you have a tutorial group on 14 Oct, *not* on 21 Oct. Your first tutorial group meeting (next week, 7 Oct) is listed correctly.

Overview of Week 3

This week, we will switch from language models for scoring and generating sequences of words to a different kind of task: classification, where we are given some text and need to label it according to some predefined categories (such as what the topic is, or whether it is spam). We will introduce multinomial logistic regression models, which are useful for this task and also provide a solid foundation for understanding neural network models (which we'll start learning about next week).

We'll spend the first two lectures discussing how these models work and how to train them, as well as some issues specific to text classification, such as what kinds of features to use and how to evaluate the results. Then we will switch gears and start to think about lexical semantics (the meanings of words), which is the first step toward modeling some of the aspects of word sequences (such as similarity) that last week's N-gram models failed at.

Lectures and Readings

Additional Materials

• Lab 2. In this week's lab you'll work with N-gram models for language identification.
• Tutorial exercise sheet 1.
  • These are the questions that will be discussed in the first tutorial group meetings next week (Week 4).
  • Unlike labs, you are expected to work through the questions in advance, before your tutorial session. During the tutorial session, you will sit at a table of 5-8 students to discuss your answers and identify any issues/questions. Tutors will be available to help answer your questions. (Of course, you can also still use Piazza or the TA help hour to ask questions, about these exercises or anything else.)
  • Answers will be made available after all groups have met (end of Week 4).
  • By now you should have received your tutorial group assignment, and it should show up in your timetable. If you have a regular conflict with the time of your tutorial (or lab) group, you may request a switch using this link.
• Preview of next week's reading. If you want to get ahead on next week's reading, you can start with these:
  • For Monday: 5.4-5.5 (*), 5.6-5.10
  • For Wednesday: 6.0-6.3.0 (*), 6.4-6.5.
    • Note: this chapter introducts neural networks in a bottom up way (from the pieces to the big picture). If you are finding it hard to follow the explanation or the maths, or just prefer to see the big picture first, you might want to do this reading after the lecture. In the lecture I am planning to focus more on the big picture first: the relationship of NNs to logistic regression, the limitations of LR that NNs are able to solve, and what allows them do that.