This project combines algebraic geometry and singular learning theory to study how Bayesian neural networks learn. Using these frameworks, we will develop the theory needed to design fast algorithms that account for the hidden structural properties, ultimately leading to more reliable Bayesian approximations and thus improved uncertainty quantification.

This project hypothesizes that the foundational framework introduced by SLT, which utilizes results from Bayesian statistics and algebraic geometry, will prove beneficial for further developments within the understanding of Bayesian neural networks (BNNs). Concretely, this project will investigate the following research objectives; (i) Characterize reparameterizations in neural networks. Utilize the mathematical framework in SLT and algebraic geometry to describe and classify reparameterizations of neural networks. (ii) Leverage reparameterizations for better learning algorithms. Use this framework to design new optimization and inference methods that explicitly account for reparameterization structure, to improve the reliability of BNNs. (iii) Clarify the link between stochastic gradient descent and Bayesian inference. Investigate how the solutions found by SGD relate to the Bayesian posterior in nonidentifiable models, and explore how this connection is influenced by singularities and reparameterizations.

If the project is successful, it will yield both theoretical and practical contributions at the intersection of mathematics and machine learning. On the theoretical side, it will advance the understanding of reparameterizations and singularities in BNNs. Notably, it will develop an algorithm to quantify reparameterizations for a neural network given the architecture. The theory will translate into novel algorithms for optimization and variational inference by enabling reparameterization-aware procedures sensitive to the underlying geometry. Further, the research will clarify the relationship between stochastic gradient descent and Bayesian posteriors in non-identifiable models, offering an understanding of the conceptual link between practical training methods and the statistical theory.

These developments will contribute to the construction of safer and more trustworthy Al systems. Improved uncertainty quantification and more robust inference methods are particularly relevant in domains such as healthcare, finance, and drug discovery, where overconfident predictions can have severe consequences.