Week 3: Classification and lexical semantics
Reminders and Announcements
Welcome to the start of our third week! We are seeing some good questions on EdStem (and thanks especially to those of you who have answered other students' questions!), and we hope you all enjoyed the first lab! Our demonstrators thought it went well, but if you do have any suggestions on how to improve it for next year, please let us know. If you are new to Edinburgh, I hope you're also getting to know the city a little bit.
I (SG) am back to my duties as course organizer this week. This page went live last week with some out of date information from last year which I've now updated after coming off strike. I do hope you were able to follow the video lectures, as last week's material does feed into a lot of what follows, and will also be covered in the labs this week!
Here are some reminders for this week:
- We have our second lab this week, and then we'll switch to tutorial groups for Weeks 4 and 5.
- Tutorial groups do require preparation -- see under Additional Materials below.
Overview of Week 3
This week, we will switch from language models for scoring and generating sequences of words to a different kind of task: classification, where we are given some text and need to label it according to some predefined categories (such as what the topic is, or whether it is spam). We will introduce multinomial logistic regression models, which are useful for this task and also provide a solid foundation for understanding neural network models (which we'll start learning about next week).
We'll spend the first two lectures discussing how these models work and how to train them, as well as some issues specific to text classification, such as what kinds of features to use and how to evaluate the results. Then we will switch gears and start to think about lexical semantics (the meanings of words), which is the first step toward modelling some of the aspects of word sequences (such as similarity) that last week's N-gram models failed at.
Lectures and Readings
| Lecture # | Who? | Slides | Reading |
|---|---|---|---|
| 1 | EA | Text classification with logistic regresssion | JM3 4.0-4.4 (all * except 4.3.2-4.3.3) Known typos:
|
| 2 | EA | Training logistic regression and evaluating classification | JM3 4.5-4.7.0 (*), 4.7.1-4.8, 4.9 (*) |
| 3 | EA | Lexical semantics | JM3 5.1-5.3 (*), Appendix I.0-I.3 |
Additional Materials
- Lab 2. In this week's lab you'll work with N-gram models for language identification.
- Tutorial exercise sheet 1.
- These are the questions that will be discussed in the first tutorial group meetings next week (Week 4).
- Unlike labs, you are expected to work through the questions in advance, before your tutorial session. During the tutorial session, you will sit at a table of 5-8 students to discuss your answers and identify any issues/questions. Tutors will be available to help answer your questions. (Of course, you can also still use the forum or TA help hour to ask questions, about these exercises or anything else.)
- Answers will be made available after all groups have met (end of Week 4).
- By now you should have received your tutorial group assignment, and it should show up in your timetable. If you have a regular conflict with the time of your tutorial (or lab) group, you may request a switch using this link.