Nathaël Da Costa

Deep neural networks parameterise manifolds of functions with a finite dimensional parameter spaces. From a learning perspective, these function manifolds are the true objects of interest. However deep learning algorithms act in the parameter space, and such algorithms do not solely depend on the function manifold, but also on the choice of parametrisation.

Relevant examples of such algorithms arise in uncertainty quantification. Deep networks can be made into probabilistic models, outputting distributions instead of point estimates. This can be done for example through approximate Bayesian inference over the parameter space, such as with Laplace approximations. Importantly, these approximations depend on the parametrisation of the neural network.

Differential geometry provides the language to study the effect of reparametrisations on such algorithms. The goal of this project is to improve our understanding of both training dynamics and uncertainty quantification in deep neural networks using differential geometry. To achieve this goal, novel theory will be developed. Such theory will be applied in practical algorithms with the aim at overcoming current limitations in probabilistic deep learning.