Improved null proportion estimators for multiple discrete tests with plug-in FDR control

Abstract

It is well known that the performance of the Benjamini and Hochberg (BH) procedure can be improved by incorporating estimators of the number or proportion of null hypotheses to yield an adaptive BH procedure which still controls FDR. Several such plug in estimators have been proposed. For some of these, such as Storey's estimator, plug in FDR control has been established, while for some others, such as the Pounds and Cheng estimator, some gaps remain to be closed. These developments have largely focused on the case of continuous test statistics, where null p values follow the uniform distribution. In the discrete setting, although these estimators continue to provide plug in FDR control, they become overly conservative, leading to inefficient procedures. In this paper, a general class of estimators that encompasses the classical Storey and Pounds and Cheng estimators is introduced. Alongside, several generic strategies to mitigate conservativeness in the discrete setting are proposed by incorporating information about the null distribution functions. These strategies provably yield less conservative estimates while maintaining valid FDR control, and the resulting performance gains are illustrated on both real and simulated data. As a byproduct of a more general result, plug in FDR control for the Pounds and Cheng estimator in the continuous case is also established.