Bayesian inference in machine learning

Statistical inference for binary data is one of the central problem in machine learning which can be treated within a nonparametric Bernoulli model. In many situations the underlying Bernoulli parameter may approach the edges zero or one that leads to inference for non-regular models. Properties like asymptotic normality of the MLE or of the posterior distribution are not valid in such irregular cases. The approach of this paper suggests to impose a Gaussian prior on the canonical Bernoulli parameter, or, analogously, to consider a penalized MLE with a quadratic penalty. A Gaussian prior can be viewed as a kind of a barrier preventing the canonical parameter to approach infinity and forcing it to stay within the region of regularity. The main results stay the classical results like asymptotic normality of the posterior distribution or asymptotic normality of the penalized MLE for any configuration of Bernoulli parameters under the assumption that the prior is strong enough in direction. We also demonstrate how the results can be applied to analysis of random graphs, ranking problems, and nonparametric binary classification.

***This seminar takes place within the series Research Seminars Mathematical Statistics and due to the current situtation will be held online. Please contact liv.heinecke[at]uni-potsdam.de to receive the zoom link.***