Random matrix theory is a large area with a long history, with elegant theory and a wide range of applications. However, the challenges of modern machine learning are forcing us not only to use random matrix theory in new ways, and also to chart out new directions for theory. In this series of presentations, we’ll cover several aspects of these developments. This includes challenges in training machine learning models, inference in overparameterized models, diagnostics where heavy-tailed distributions are ubiquitous, and computational-statistical tradeoffs in randomized numerical linear algebra. Addressing these challenges leads to new directions for theory: phenomenology and semi-empirical theory to characterize performance in state-of-the-art neural networks without access to training or testing data; high-dimensional linearizations and deterministic equivalents to go beyond eigenvalue distributions of linear models; very sparse embeddings to perform “algorithmic gaussianization” to speed up core numerical linear algebra problems; new random matrix models that have heavy-tailed spectral structure without having heavy-tailed elements; and using “free compression” ideas in reverse to compute high-quality spectral distributions of so-called impalpable matrices (for which we cannot form or even evaluate with full matrix-vector products).
Bio:
Michael W. Mahoney is at the University of California at Berkeley in the Department of Statistics and at the International Computer Science Institute (ICSI). He is also an Amazon Scholar as well as head of the Machine Learning and Analytics Group at the Lawrence Berkeley National Laboratory. He works on algorithmic and statistical aspects of modern large-scale data analysis. Much of his recent research has focused on large-scale machine learning, including randomized matrix algorithms and randomized numerical linear algebra, scientific machine learning, scalable stochastic optimization, geometric network analysis tools for structure extraction in large informatics graphs, scalable implicit regularization methods, computational methods for neural network analysis, physics informed machine learning, and applications in genetics, astronomy, medical imaging, social network analysis, and internet data analysis. He received his PhD from Yale University with a dissertation in computational statistical mechanics, and he has worked and taught at Yale University in the mathematics department, at Yahoo Research, and at Stanford University in the mathematics department. Among other things, he was on the national advisory committee of the Statistical and Applied Mathematical Sciences Institute (SAMSI), he was on the National Research Council’s Committee on the Analysis of Massive Data, he co-organized the Simons Institute’s fall 2013 and 2018 programs on the foundations of data science, he ran the Park City Mathematics Institute’s 2016 PCMI Summer Session on The Mathematics of Data, he ran the biennial MMDS Workshops on Algorithms for Modern Massive Data Sets, and he was the Director of the NSF/TRIPODS-funded FODA (Foundations of Data Analysis) Institute at UC Berkeley. More information is available at https://www.stat.berkeley.edu/~mmahoney/.