Betting on Bets: Anytime-Valid Tests for Stochastic Dominance

How can we monitor, in real time, whether one uncertain prospect has any upside over another?

To answer this question, we develop a novel family of sequential, anytime-valid tests for stochastic dominance (SD), a classical and popular notion for comparing entire distribution functions.

The problem is distinct from that of testing mean dominance, and it is particularly useful when comparing distributions with similar means or with ordinal outcomes.

We first derive powerful, nonparametric e-processes that quantify evidence against the null hypothesis that one prospect is dominated by another.

For first-order SD, these e-processes are mixtures of asymptotically growth-rate optimal e-variables, yielding a test of power one that retains validity under continuous monitoring.

The overall approach further generalizes to sequential testing for higher-order SD and other integral stochastic orders.

Empirically, we show that the tests are competitive in power with classical, non-anytime-valid SD tests.

We also present a real-world application to baseball analytics, examining a controversial phenomenon known as third time-through-the-order penalty for starting pitchers.

We close by sketching the complementary problem of testing whether a prospect has a definite upside, and formalize the conditions under which we can derive a nontrivial anytime-valid test.