Showing 1–2 of 2 results for author: Unlu, A

Variational Laplace for Bayesian neural networks

Authors: Ali Unlu, Laurence Aitchison

Abstract: We develop variational Laplace for Bayesian neural networks (BNNs) which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is (in essence) the log-likelihood, plus weight-decay, plus a squared-gradient regularizer. Variational L… ▽ More We develop variational Laplace for Bayesian neural networks (BNNs) which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is (in essence) the log-likelihood, plus weight-decay, plus a squared-gradient regularizer. Variational Laplace gave better test performance and expected calibration errors than maximum a-posteriori inference and standard sampling-based variational inference, despite using the same variational approximate posterior. Finally, we emphasise care needed in benchmarking standard VI as there is a risk of stop** before the variance parameters have converged. We show that early-stop** can be avoided by increasing the learning rate for the variance parameters. △ Less

Submitted 20 July, 2021; v1 submitted 27 February, 2021; originally announced March 2021.

Comments: Accidental resubmission of new version of arXiv:2011.10443

Variational Laplace for Bayesian neural networks

Authors: Ali Unlu, Laurence Aitchison

Abstract: We develop variational Laplace for Bayesian neural networks (BNNs) which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is (in essence) the log-likelihood, plus weight-decay, plus a squared-gradient regularizer. Variational L… ▽ More We develop variational Laplace for Bayesian neural networks (BNNs) which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is (in essence) the log-likelihood, plus weight-decay, plus a squared-gradient regularizer. Variational Laplace gave better test performance and expected calibration errors than maximum a-posteriori inference and standard sampling-based variational inference, despite using the same variational approximate posterior. Finally, we emphasise care needed in benchmarking standard VI as there is a risk of stop** before the variance parameters have converged. We show that early-stop** can be avoided by increasing the learning rate for the variance parameters. △ Less

Submitted 10 August, 2021; v1 submitted 20 November, 2020; originally announced November 2020.