Estimation and Confidence Sets for Sparse Normal Mixtures
Tony Cai, Jiashun Jin, and Mark Low
-
Abstract:
For high dimensional statistical models, researchers
have begun to focus on situations which can
be described as having relatively few
moderately large coefficients.
Such situations lead to some very subtle statistical
problems. In particular, Ingster and Donoho and Jin
have considered a sparse normal means testing
problem, in which they described the precise demarcation, or the
detection boundary.
Meinshausen and Rice have shown that
it is even possible to estimate consistently the fraction
of nonzero coordinates on a subset of
the detectable region, but leave unanswered the question of
exactly which parts of the detectable region that consistent
estimation is possible.
In the present paper we develop a new approach for estimating the fraction of nonzero means for problems where the nonzero means are moderately large. We show that the detection region described by Ingster and Donoho and Jin turns out to be the region where it is possible to consistently estimate the expected fraction of nonzero coordinates. This theory is developed further and minimax rates of convergence are derived. A procedure is constructed which attains the optimal rate of convergence in this setting. Furthermore, the procedure also provides an honest lower bound for confidence intervals while minimizing the expected length of such an interval. Simulations are used to enable comparison with the work of Meinshausen and Rice, where a procedure is given but where rates of convergence have not been discussed. Extensions to more general Gaussian mixture models are also given.
- Paper:
pdf file.
- Other related papers:
Jin, J. & Cai, T. (2007).
Estimating the null and the proportion of non-null effects in large-scale multiple comparisons.
J. American Statistical Association 102, 495-506. - Sun, W. & Cai, T. (2007).
Oracle and adaptive compound decision rules for false discovery rate control.
J. American Statistical Association 102, 901-912.