SILO: Optimal vintage factor analysis with deflation varimax

Vintage factor analysis is one important type of factor analysis that aims to first find a low-dimensional representation of the original data, and then to seek a rotation such that the rotated low-dimensional representation is scientifically meaningful. The most widely used vintage factor analysis is the Principal Component Analysis (PCA) followed by the varimax rotation. Despite its popularity, little theoretical guarantee can be provided to date mainly because varimax rotation requires solving a non-convex optimization over the set of orthogonal matrices.
We propose a deflation varimax procedure that solves each row of an orthogonal matrix sequentially. In addition to its net computational gain and flexibility, we are able to fully establish theoretical guarantees for the proposed procedure in a broader context. Adopting this new deflation varimax as the second step after PCA, we further analyze this two-step procedure under a general class of factor models. Our results show that it estimates the factor loading matrix in the minimax optimal rate when the signal-to-noise-ratio (SNR) is moderate or large. In the low SNR regime, we offer possible improvement over using PCA and the deflation varimax when the additive noise under the factor model is structured. The modified procedure is shown to be minimax optimal in all SNR regimes.

Yuqian Zhang is an Assistant Professor in the Department of Electrical and Computer Engineering at Rutgers University. She received her PhD in Electrical Engineering from Columbia University in 2018. Before joining Rutgers, she was a postdoc fellow with Cornell Tripods Data Science Center from 2018-2019. Her research interests include machine learning, nonconvex optimization, and application in scientific data processing.