Fitting high-dimensional linear models by M-estimation: some surprising asymptotic phenomena

This talk reviews some recent work on (unpenalized) linear re- gression M-estimators in high-dimensions. Extending the seminal work of Peter Huber, Steve Portnoy and others to the setting where n, the number of ob- servations, is large and comparable to p, the number of predictors, we obtain updated results for the asymptotic statistical behavior of the estimates. Some surprising phenomena are revealed, including:
1. The maximum likelihood estimate is generally suboptimal in terms of asymptotic variance;
2. The optimal objective function amongst all M-estimates can be computed, and it depends on certain aspects of the statistical model as well as the limit of the ratio p/n.
I will also present some extensions to penalized regression M-estimates. This talk covers joint work with Noureddine El Karoui, Peter Bickel, Chinghway Lim, and Bin Yu.