Trace-class Gaussian priors for Bayesian learning of neural networks with MCMC

I will discuss a new neural network based prior for real valued functions on ℝ^d. The new prior is a Gaussian neural network prior, where each weight and bias has an independent Gaussian prior, but with the key difference that the variances decrease in the width of the network in such a way that the resulting function is almost surely well defined in the limit of an infinite width network.
In the talk, I will firstly motivate the work by looking at stochastic control problems, where classic Karhunen-Loève priors struggle to scale up to the domain sizes we are interested in. I will then give some background and highlight connections to other function priors, before defining the new trace-class neural network (tcNN) prior. Benefits and disadvantages of the different priors will be discussed, and the advantages of tcNN priors over other function space priors will be highlighted in numerical examples. I will end the talk with a discussion of open questions and explore links to research undertaken at the SFB.
Reference: Trace-class Gaussian priors for Bayesian learning of neural networks with MCMC (2021), Torben Sell and Sumeetpal S Singh, arXiv preprint arxiv.org/abs/2012.10943v2, under review