Semiparametric Efficiency in Sequential Experiments: Characterization and Design via Average Propensity

Modern experiments, including evaluations of AI-enabled services and platform interventions, often depart from independent and identically distributed (i.i.d.) sampling because assignments may be adaptive, balanced across covariates, or subject to rollout constraints such as exposure, fairness, and budget limits.

This paper studies the efficiency benchmark for estimating causal targets in such sequential experiments.

We show that every non-anticipating design induces an average propensity score, and we establish a semiparametric lower bound: for regular locally unbiased estimators, attainable precision is bounded by the i.i.d. efficiency benchmark evaluated at this induced score.

The average propensity score thereby serves as a common benchmark and design target, allowing sequential experimental design to be viewed as choosing or learning an efficient allocation rule, with operational constraints entering through the admissible set when present.

We then develop implementable batched adaptive designs that approach this benchmark through two complementary mechanisms.

The first uses regression adjustment based on efficient influence functions; for general smooth estimands it attains the benchmark under standard nuisance-rate conditions, while for linear functionals of outcome means it achieves a sharp second-order rate.

The second uses adaptive covariate balancing to attain the same benchmark through the assignment mechanism, enabling simple moment-based estimation.

Both routes require only a small number of policy updates, making them compatible with delayed feedback and easier to monitor in operational deployments.

Numerical experiments and an empirical study of AI medical-assistant evaluation demonstrate the practical efficiency gains, including in multi-treatment settings.

Overall, the paper provides a unified framework for characterizing and designing efficient sequential experiments.