Lectures on Ultra High Dimensional Regression
at Department of Biostatistics, Harvard University, April 16, 2010
The analysis of high-dimensional data now commonly arising in scientific investigations poses many statistical challenges not present in smaller scale studies. In these lectures I will discuss high-dimensional linear regression with large p and small n. This problem has attracted much recent interest in a number of fields including applied mathematics, electrical engineering, and statistics.
To provide a proper background and foundation for the main topics, we shall begin with discussions on the high-dimensional Gaussian sequence model. We then consider the linear model y = Xβ + z, where the dimension of the signal β is much larger than the number of observations. It is now well understood that l1 minimization methods provide effective ways for high dimensional sparse regression. I will present an elementary and unified analysis of l1 minimization methods including Lasso and the Dantzig Selector in three settings: noiseless, bounded error and Gaussian noise. Time permitting, I will also discuss l1 minimization approaches to sparse precision matrix estimation.