Sparse PCA: Optimal Rates and Adaptive Estimation
Tony Cai, Zongming Ma, and Yihong Wu
-
Abstract:
Principal component analysis (PCA) is one of the most commonly used statistical procedures with a wide range of applications.This paper considers both minimax and adaptive estimation of the principal subspace in the high dimensional setting. Under mild technical conditions, we first establish the optimal rates of convergence for estimating the principal subspace which are sharp with respect to all the parameters, thus providing a complete characterization of the difficulty of the estimation problem in term of the convergence rate. The lower bound is obtained by calculating the local metric entropy and an application of Fano's Lemma. The rate optimal estimator is constructed using aggregation, which, however, might not be computationally feasible.
We then introduce an adaptive procedure for estimating the principal subspace which is fully data driven and can be computed efficiently. It is shown that the estimator attains the optimal rates of convergence simultaneously over a large collection of the parameter spaces. A key idea in our construction is a reduction scheme which reduces the sparse PCA problem to a high-dimensional multivariate regression problem. This method is potentially also useful for other related problems.
- Paper:
pdf file.
- Other related papers:
Cai, T., Zhang, C.-H. & Zhou, H. (2010).
Optimal rates of convergence for covariance matrix estimation
The Annals of Statistics 38, 2118-2144.Cai, T., Liu, W. & Luo, X. (2011).
A constrained l1 minimization approach to sparse precision matrix estimation.
J. American Statistical Association 106, 594-607.Cai, T. & Liu, W. (2011).
Adaptive thresholding for sparse covariance matrix estimation.
J. American Statistical Association 106, 672-684.Cai, T. & Zhou, H. (2012).
Minimax estimation of large covariance matrices under l1 norm (with discussion).
Statistica Sinica 22, 1319-1378.Cai, T. & Yuan, M. (2012).
Adaptive covariance matrix estimation through block thresholding.
The Annals of Statistics 40, 2014-2042.Cai, T. & Zhou, H. (2012).
Optimal rates of convergence for sparse covariance matrix estimation.
The Annals of Statistics 40, 2389-2420.Cai, T., Ren, Z. & Zhou, H. (2013).
Optimal rates of convergence for estimating Toeplitz covariance matrices.
Probability Theory and Related Fields 156, 101-143.Cai, T., Ma, Z. & Wu, Y. (2013).
Optimal estimation and rank detection for sparse spiked covariance matrices.
Technical report.