Post-Doc in Machine Learning and Natural Language Processing

Large-scale language models have led to impressive results in many NLP tasks, exhibiting transfer and few-shot learning capabilities. When interacting with such systems, users commonly find them capable of reasoning, planning, and explaining their decisions, often in convincing ways. However, despite the enormous advances in the last years, current deep learning models for NLP are still very limited in fundamental ways and many important ingredients are still missing to achieve a satisfactory level of "intelligence". Some of these limitations partly stem from their monolithic architectures, which are good for some perceptual tasks, but unsuitable for tasks requiring higher-level cognition.

The overarching goal of DECOLLAGE is to attack these fundamental problems by bringing together tools and ideas from machine learning, sparse modeling, information theory, and cognitive science, in an interdisciplinary approach.

Four research directions are:

1. Designing new components for utility guidance, control, and contextualization. This will endow the model with the ability to predict its own quality (an "inner voice" or a critic) and to handle contextual information (e.g. document-level, conversation-level, meta-information about the surrounding environment), doing so in a modular, selective, and efficient manner.
2. Developing dynamic memory structures that facilitate continual learning, by supporting efficient reading and writing access, fast adaptation, and representation of world and self-knowledge. We will exploit synergies with sparse modeling and information retrieval.
3. Formalizing and implementing new mathematical models for sparse communication, bridging the gap between discrete (symbolic) and continuous representations, and developing techniques to integrate multiple modalities (such as text, speech, and image signals) into a shared representation space. This will draw links between information theory, formal languages, and neuroscience.
4. Develop new methods for automatic and semi-automatic evaluation of multimodal language models and their various components in multiple dimensions.

The postdoc is a 2-year position, with the possibility of extension up to 3 years. The salary range is 2500-3000 EUR (free of tax), depending on the candidate's experience. These positions can potentially evolve into "ELLIS Post-Docs" and include a visit to an exchange institution in the ELLIS network.

See here for more information about the SARDINE Lab and some of our recent publications.

If interested in this position, please send an email with your CV to andre.t.martins AT tecnico.ulisboa.pt. 

Apply by October 31st.