Variance Deltas for Visualizing and Explaining Posterior Uncertainty

In observational settings, where the data generating process and possibly the sample size are not controlled, it is typically impossible to guarantee a priori that quantities of interest will be estimated with sufficient precision.

However, even when the data do not determine the quantities of interest, they may still allow determination of what is missing -- unobserved information which, if observed, would meaningfully reduce uncertainty.

We propose an interactive visualization system, termed variance deltas, to enable the discovery of such missing information from a Bayesian posterior distribution.

This system, which we provide as a software package, represents missing information as subsets of unobserved model quantities, organized into a tree based on how well each subset explains uncertainty about the quantity of interest.

This system both automates the construction of candidate subsets from minimal user input and implements interactive operations for the division and combination of subsets, allowing the efficient discovery of interesting and useful explanations.

We demonstrate this system by using it to discover nonobvious explanations of uncertainty for (1) a treatment effect parameter in a simulated causal inference problem and (2) a population proportion in a forecasting model of real polling data with many sources of bias.