The Randomized BH Procedure: A Generalized Framework Encompassing Conformal and Competition Tests

We propose the Randomized BH procedure, a generalized framework that introduces randomization into the Benjamini-Hochberg (BH) procedure for false discovery rate (FDR) control.

We show that the classical BH procedure as well as two recently developed families of null-distribution-free methods, competition tests and conformal tests, share a common theoretical foundation and arise as special cases of this framework.

The Randomized BH relaxes the classical conditions required for FDR control, frees conformal tests from the strict exchangeability assumption while allowing calibration sets to be constructed via pre-screening, and enables competition tests to be analyzed through the well-developed theory of p-values.

We illustrate the framework through two applications.

The first is a deeper analysis of compound p-values through the lens of randomization.

The second is a general approach for integrating distributed multiple testing problems while maintaining global FDR control.

Numerical simulations confirm the validity of our theoretical results.