ANLP: Schedule and Course Materials

Below is the planned schedule and list of topics for 2025-26, but details may change.

Links for Weeks1-11 will start working on the Sunday morning at the start of each week.

• Week <1: Preparation steps to take. Available now! Please visit this page to help you prepare for the course.
• Week 1: Language as data: structure and statistics. Including: levels of structure and ambiguity, corpora, Zipf's law, morphology across languages, BPE tokenization.
• Week 2: N-gram models. Including: models and parameters, n-gram language models, training and evaluating, smoothing, and sampling.
• Week 3: Classification and lexical semantics Including: examples of text classification, multinomial logistic regression models and training, word senses and relations, WordNet, distributional semantics.
• Week 4: Word embeddings and neural networks Including: Dense word embeddings (word2vec), semantic similarity measures and evaluation, linear separability, multi-layer perceptron model and training.
• Week 5: Algorithmic bias, language variation, and RNNs. Including: Algorithmic bias, protected characteristics, dialects and varieties, linguistic discrimination; recurrent neural network language models, long-distance dependencies.
• Week 6: Attention and Transformers Including: Sequence-to-sequence RNNs with attention, self-attention and Transformer blocks, parallelization, Transformer architectures (encoder, decoder, encoder-decoder).
• Week 7: Transfer learning and LLMs Including: positional embedding, LLM pretraining and fine-tuning, BERT, GPT, T5
• Week 8: LLM Inference and Scaling Laws Including: in-context learning, chain-of-thought reasoning, scaling laws.
• Week 9: Post-training: Instruction Tuning and Reinforcement Learning  Including: Instruction tuning, reinforcement learning with human feedback and from verifiable rewards.
• Week 10: Guest lectures and exam preparation
• Weeks 11+: Exam revision No new material.