Operator-valued models, like neural operators, prior-fitted networks, and neural posterior estimation, are a recent big trend in machine learning. These are models that learn to generalize between learning tasks, to solve the "zero-shot" problem of returning a "trained" model or posterior distribution from a given dataset. They are often motivated through the idea of amortization: It may make sense to invest significant computational resources into training such a foundation model, if it is then able to solve a large number of challenging inference problems without actually solving an explicit inference problem.
In the PhD, we want to (at least initially) explore fundamental theoretical limitations of Amortization as a concept. Since probabilistic inference is a very general operation, it includes computational tasks of known worst-case and average-case complexity that differ explicitly from the runtime complexity of the foundation models mentioned above. There must therefore be inference problems that cannot be learned, or in which amortization breaks down in some form. As amortized models are now increasingly applied across industry and science, it is important to understand such limitations to avoid nasty surprises.
