Felicia Körner

Recent advances in Natural Language Processing (NLP) have led to the widespread adoption of large language models. While these models excel at many NLP tasks in high-resource languages, their performance in less well-resourced languages lags behind. To address this disparity, different methods to transfer knowledge from high- to low-resource settings have been explored. One promising approach is to combine task- and language-specific modules such that their task expertise or language proficiency can complement one another. This approach fosters cross-lingual transfer and allows for more adaptable, efficient systems. It also has the potential to be more scalable and sustainable than training from scratch, as existing models can be reused and combined. In parallel, recent work has looked into what gives models multilingual ability and how models represent linguistic features, enabling a better understanding of how the performance gap between languages can be bridged. We plan to further investigate mechanisms of multilingualism in models, and, using these insights, explore how models can be combined to transfer and share knowledge. Ultimately, our aim is to develop models that are more sample-efficient and robust to cross-lingual variation, improving performance in low-resource settings while preserving capabilities in high-resource settings.