Felix Sarnthein

My proposed PhD research will be carried out in the area of Deep Learning. In particular, I am interested in understanding the interaction of parameterization, objective, and optimization as fundamental principles of deep learning. While great empirical progress on these questions has been achieved in various domains, modeling of long-range interactions still poses an unsolved challenge, and a comprehensive theory of learning is still to be devised. During my PhD, I want to contribute to the understanding and development of potentially self-supervised methods to model long-range interactions. In detail, I plan to investigate the parameterization, initialization, objectives and optimization of a novel class of linear recurrent neural networks to achieve long-range representation learning. This is relevant for many applications which require global and hierarchical processing of long sequence data such as audio or DNA.