Optimal Detection For Sparse Mixtures
Tony Cai and Yihong Wu
-
Abstract:
Detection of sparse signals arises in a wide range of modern scientific studies. The focus so far has been mainly on Gaussian mixture models. In this paper, we consider the detection problem under a general sparse mixture model and obtain an explicit expression for the detection boundary. It is shown that the fundamental limits of detection is governed by the behavior of the log-likelihood ratio evaluated at an appropriate quantile of the null distribution. We also establish the adaptive optimality of the higher criticism procedure across all sparse mixtures satisfying certain mild regularity conditions. In particular, the general results obtained in this paper recover and extend in a unified manner the previously known results on sparse detection far beyond the conventional Gaussian model and other exponential families.
- Paper:
pdf file.
- Other related papers:
Cai, T. Jin, J. & Low, M. (2007).
Estimation and confidence sets for sparse normal mixtures.
The Annals of Statistics 35, 2421-2449.Jin, J. & Cai, T. (2007).
Estimating the null and the proportion of non-null effects in large-scale multiple comparisons.
J. American Statistical Association 102, 495-506.Cai, T. & Jin, J. (2010).
Optimal rates of convergence for estimating the null and proportion of non-null effects in large-scale multiple testing.
The Annals of Statistics 38, 100-145.Cai, T., Jeng, J. & Jin, J. (2011).
Optimal detection of heterogeneous and heteroscedastic mixtures.
Journal of the Royal Statistical Society, Series B 73, 629-662.