When Does Real-World Data Make a Randomized Trial More Efficient, and How Would You Know? A Report Card, an Efficiency Map, and Selection-Aware Inference for Adaptive-TMLE

Augmenting a randomized controlled trial with real-world data promises greater efficiency, but how much a given fusion actually delivers, and how to attach honest uncertainty to that gain, is rarely characterized.

Using adaptive targeted maximum likelihood estimation (A-TMLE) as the running example, we develop three reproducible tools for honest evidence from combined trial and real-world data.

First, a report card that makes the estimator's data-adaptively learned bias model auditable, measuring how well it recovers the true enrollment-effect surface and attributing the estimator's variance to its structural parts.

Second, a map of when fusion helps versus hurts, benchmarked against an efficient trial-only estimator: the gain is driven primarily by the magnitude of the real-world bias rather than its functional complexity, a dominance an exact variance identity explains; it crosses break-even near a moderate bias and erodes as the trial grows, so the advantage is finite-sample rather than a form of super-efficiency.

Third, selection-aware inference for the gain, treated as a data-adaptive estimand: the naive standard error undercovers, and among ten candidate standard errors only a block jackknife is calibrated, though conservatively so.

Three openly available fusions, in a biomedical HIV trial, a public-health trial, and a job-training trial, span the map and show the difference an honest interval makes for real-world evidence.