Compressed Sensing and High-Dimensional Linear Regression
NCMIS Distinguished Lectures
Presented at the Chinese Academy of Sciences, Beijing, June 2014
Abstract: These lectures will focus on compressed sensing and high dimensional linear regression. These and other related problems have attracted much recent interest in a range of fields including statistics, machine learning and electrical engineering. In the high dimensional setting where the dimension p can be much larger than the sample size n, classical methods and results based on fixed p and large n are no longer applicable. We will analyze in detail the constrained and penalized l1 minimization methods for compressed sensing/high-dimensional regression and give a unified and elementary analysis on sparse signal recovery in three settings: noiseless, bounded noise and Gaussian noise. Johnson-Lindenstrauss Lemma and construction of compressed sensing matrices will also be discussed. In addition, we will discuss at the end some recent results on optimal and adaptive estimation of high-dimensional covariance/precision matrices under different settings.
References:
- Baraniuk, R., Davenport, M., DeVore, R. and Wakin, M. (2008). A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253-263.
- Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous Analysis of Lasso and Dantzig Selector. The Annals of Statistics 37, 1705-1732.
- C a i , T . T . and J i a n g , T . ( 2 0 1 1 ) . L i m i t i n g l a w s o f c o h e r e n c e o f r a n d o m m a t r i c e s w i t h a p p l i c a t i o n s t o t e s t i n g c o v a r i a n c e s t r u c t u r e a n d c o n s t r u c t i o n o f c o m p r e s s e d s e n s i n g m a t r i c e s . T h e A n n a l s o f S t a t i s t i c s 3 9 , 1 4 9 6 - 1 5 2 5 .
- Cai, T. T. and Liu, W. (2011). A d a p t i v e t h r e s h o l d i n g f o r s p a r s e c o v a r i a n c e m a t r i x e s t i m a t i o n . J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 1 0 6 , 6 7 2 - 6 8 4 .
- Cai, T. T. and Liu, W. (2011). A d i r e c t e s t i m a t i o n a p p r o a c h t o s p a r s e l i n e a r d i s c r i m i n a n t a n a l y s i s . J o u r n a l o f t h e A m e r i c a n S t a t i s t i c a l A s s o c i a t i o n 1 0 6 , 1 5 6 6 - 1 5 7 7 .
- Cai, T. T., Liu, W. & Luo, X. (2011). A constrained l1 minimization approach to sparse precision matrix estimation. J. American Statistical Association 106, 594-607.
- Cai, T. T., Liu, W. and Zhou, H. (2012). E s t i m a t i n g s p a r s e p r e c i s i o n m a t r i x : O p t i m a l r a t e s o f c o n v e r g e n c e a n d a d a p t i v e e s t i m a t i o n . T h e A n n a l s o f S t a t i s t i c s , t o a p p e a r .
- Cai, T. T., Wang, L. & Xu, G. (2010). Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing 58, 1300-1308.
- Cai, T. T., Wang, L. & Xu, G. (2010). Stable recovery of sparse signals and an oracle inequality. IEEE Transactions on Information Theory 56, 3516-3522.
- Cai, T. T. and Yuan, M. (2012). A d a p t i v e c o v a r i a n c e m a t r i x e s t i m a t i o n t h r o u g h b l o c k t h r e s h o l d i n g . T h e A n n a l s o f S t a t i s t i c s 4 0 , 2 0 1 4 - 2 0 4 2 .
- C a i , T . T . and Z h a n g , A . ( 2 0 1 4 ) . S p a r s e r e p r e s e n t a t i o n o f a p o l y t o p e a n d r e c o v e r y o f s p a r s e s i g n a l s a n d l o w - r a n k m a t r i c e s . I E E E T r a n s a c t i o n s o n I n f o r m a t i o n T h e o r y 6 0 , 1 2 2 - 1 3 2 .
- Cai, T., Zhang, C.-H. & Zhou, H. (2010). Optimal rates of convergence for covariance matrix estimation. The Annals of Statistics 38, 2118-2144.
- Cai, T. and Zhou, H. (2012). O p t i m a l r a t e s o f c o n v e r g e n c e f o r s p a r s e c o v a r i a n c e m a t r i x e s t i m a t i o n . T h e A n n a l s o f S t a t i s t i c s 4 0 , 2 3 8 9 - 2 4 2 0 .
- Candes, E. T. and Tao, T. (2007). The Dantzig Selector: Statistical Estimation when p is much larger than n (with discussion). The Annals of Statistics 35, 2313-2351.