Blagovest Gospodinov

Despite the recent rise of machine learning, its deployment to certain high stakes sectors (healthcare, finance, automotive, etc.) has remained hampered by the insufficient reliability of many current models. At the same time, the Bayesian framework has long been identified as one of the most promising approaches to reliable machine learning. This is due to its mathematical formulation in terms of distributions rather than unqualified point estimates. It may thus seem strange that Bayesian models have not become widely deployed in practice.

While some of the reasons lie in the relative lack of familiarity with Bayes amongst practitioners, there exist two more fundamental challenges. The first is that the greater modelling power of Bayes comes at a computational price. This has resulted in a trade-off between reliability and capability. Driven by competitive market forces, practitioners have so far sided with capability and allocated their computational resources accordingly. The second challenge is related and it concerns purely algorithmic developments. While Bayesian approaches are more powerful in principle, their relative lack of popularity amongst practioners has resulted in a relatively slower research progress.

My ELLIS project aims at improving the competitiveness of Bayesian models by improving upon the efficiency of existing Bayesian models to popular problems. The hope is that by addressing the two challenges above, practitioners will be able to achieve reliability without sacrificing capability. More concretely, two problems of particular interest will be "approximate Bayesian inference" and "continual learning". The first one concerns the efficient computation of various posterior statistics and is applicable to virtually all Bayesian models. Therefore, any progress in this area is certain to push the field forward and make Bayesian models more competitive. The second problem concerns the ability of models to efficiently learn from streaming data without forgetting previously learned lessons. Progress here will mean that large models can be continously trained and adapted while preserving reliability not least in the sense of knowledge retention. This ability is likely to play a crucial role in the development of real-time models for application in automotive, robotics, etc. where a large amount of data is received constantly usually without the ability to store it long-term.