The Bias in Bayes and How to Measure it

Abstract

A Bayes prior with a likelihood can give approximate confidence and provide a remarkably flexible approach to statistical inference; but is also known to provide inaccurate perhaps incorrect results. We develop a measure of Bayes bias, first examining a simple Normal model and then progressing to quite general models with scalar and vector parameters. The Bias measure can be interpreted as the lateral displacement of the location standardized likelihood function and thus provides ready access to the effect of a prior on p-values, confidence bounds, and Bayes posterior bounds. The needed computation is comparable to that for the likelihood function and thus provides an initial option for checking merits of Bayesian computation for high dimensions.