ANLP: Schedule and Course Materials

This is the planned schedule for 2026-27. Details may change.

Each week's page will become available on the Thursday morning prior to that week.

• Week <1: Preparation steps to take. Available now! Please visit this page to help you prepare for the course.
• Week 1: Language as data: structure and statistics. Including: levels of structure and ambiguity, corpora, Zipf's law, morphology across languages, BPE tokenization.
• Week 2: N-gram models. Including: models and parameters, n-gram language models, training and evaluating, smoothing, and sampling.
• Week 3: Classification and lexical semantics Including: examples of text classification, multinomial logistic regression models and training, word senses and relations, WordNet, distributional semantics.
• Week 4: Word embeddings and neural networks Including: Dense word embeddings (word2vec), semantic similarity measures and evaluation, linear separability, multi-layer perceptron model and training.
• Week 5: Algorithmic bias, language variation, and RNNs. Including: Algorithmic bias, protected characteristics, dialects and varieties, linguistic discrimination; recurrent neural network language models, long-distance dependencies.
• Week 6: Attention and Transformers Including: Sequence-to-sequence RNNs with attention, self-attention and Transformer blocks, parallelization, Transformer architectures (encoder, decoder, encoder-decoder).
• Week 7: Transformers (cont.) and exam prep Including: transformer input/output, positional embeddings, exam revision guidance, and likely guest lecture (TBD).
• Week 8: Transfer learning and LLMs Including: LLM pretraining and fine-tuning, BERT, GPT, T5, in-context learning, chain-of-thought reasoning.
• Week 9: Multilingual models and Scaling Laws and likely guest lecture (TBD).
• Week 10: Post-training Including: Instruction tuning, reinforcement learning with human feedback and from verifiable rewards.
• Weeks 11+: Exam revision No new material.