Calibrated Probability Forecast Sequences and Measure-Valued Martingales

We consider the calibration of probability forecasts.

Several notions of calibration exist when the forecaster issues a single forecast for each of the observations that is to be predicted.

We extend one of these notions, auto-calibration, to the common situation in which the forecaster issues a sequence of forecasts for each observation, repeatedly updating their prediction as they receive additional information.

For observations that sit in any Borel space, we show that auto-calibration is equivalent to a certain sequence of random probability measures satisfying the martingale property, and we propose a simple, statistical approach to testing this property.

This provides, for the first time, a way of testing the calibration of such sequences of probability forecasts.