IRR: Natural Language Processing (Computational Linguistics)

Natural Language Processing (Computational Linguistics)

What differentiates your programme area from the other programmes on the Informatics MSc portfolio?

Natural Language Processing (NLP) is the computational study of human language and aims to answer questions such as: how can we understand, process and manipulate what people are saying or writing? Typical applications of NLP are speech recognition, machine translation, dialogue systems, text-to-speech, and information extraction. 
Apart from applications that can be deployed in industry, the same methods can be used more broadly to understand how people learn, how people think and how people interact. NLP covers a very broad area of research, drawing from linguistics, artificial intelligence, computer science and machine learning. 

Are there go-to sources to find reviews and perspectives on these research topics. i.e. are there any good review journals that are worth browsing to get topic overviews?

We do not have review journals, but some of the journals below do have review articles which would cover areas of interest. 

What are the key journals or conferences in the field for finding high quality research papers in these topics?

Conferences: 
ACL (Association for Computational Linguistics - and their regional conferences NAACL, EACL, AACL) 
EMNLP (Empirical Methods in Natural Language Processing) 
ICLR 
NeurIPS

Journals:
Transactions of the Association for Computational Linguistics
Computational Linguistics

 

Are there any particular high profile or rapidly growing research areas in the programme that you would suggest might be worth looking at for potential IRR themes?

• Efficient large pretrained language models: Over the last few years the NLP community has been learning how to leverage large amounts of unlabelled data in the form of English text or text from many languages. One of interesting problems still to solve is how to train and run inference on these models efficiently. 
• Few-shot learning: One of the most exciting things we can do with large pretrained language models is to use them as few-shot learners. We can fine-tune these models using very little labelled data, to then apply them to tasks such as classification or generation. There are many questions to resolve about how best to fine-tune the models for low-resource downstream tasks.  
• Multilingual learning: Most NLP resources are created in English but recently more multilingual NLP dataset are being created. Multilingual pretrained language models are allowing us to leverage annotations in English to solve tasks in other languages. 
• Bias and Fairness: ML models capture and can sometimes even increase the biases that are seen in natural language. There is a lot of interest in seeing how to measure and mitigate effects of bias in NLP models. 
• Adaptation: In real world applications there can be sudden shifts in the type of data that is seen by the models. Making models robust to domain shifts is important to ensure that applications can maintain their performance in the long term. 
• Compositional generalization: Humans are capable of compositional generalization , i.e.  they are able to understand novel combinations of known primitives. It constrast, it has been observed that standard neural networks cannot make sense of novel combinations of words, even if exposed to both the words and the composition rules in training. In NLP, there is a lot of interest in injecting inductive biases into the models and learning objective to encourage compositional generalization.
• Neuro-symbolic models and modular networks: The focus here is to combine stengths of deep learning methods (e.g., no need for feature engineering, flexibility) and symbolic methods (e.g., they are more transparent, easy to integrate prior knowledge into). 
• Interpretability: We understand very little about the decision and memorization mechanisms in neural models. There motivates research both into making models more transparent to humans and understanding the internal mechanics of existing models. For example, one line of work relates neural representations to representations defined by linguists.
• Multimodal models: There are increasing efforts to see how to combine language with other kinds of inputs, be that images, audio, or databases. Models which can combine signals from multiple modalities can potentially allow for more human like processing of language as the text is no longer seen in isolation. 

Contributors: A Brich and I Titov

Last update: 13 October 2021