Nadezhda Ilieva

The goal is to study modern optimization methods and their interplay with the landscape of deep neural networks, especially modern architectures such as transformers. More specifically, two directions we might take are: 1) Optimization for deep learning: how to design new methods or improve existing ones (for instance reducing the memory usage, improving the stability w.r.t hyper-parameters, dealing with more restricted settings where quantization of the parameters is required, etc) 2) Study the loss landscape of neural networks and design new neural network architectures to improve their properties.