Drifts and shifts in data distributions occur in many application domains, from computer vision to marine ecology. I will describe the notion of risk measure, dating back to seminal work by H. Scarf, which allows us to tackle the issues arising from distribution shifts in stochastic optimization and statistical prediction and train distributionally robust models. As a connecting thread through the talk, I will focus on a particularly intuitive risk measure, the superquantile or CVaR, developed by R. T. Rockafellar and collaborators, and show how a range of stochastic algorithms and statistical methods from machine learning to federated learning spawn from a smooth adaptation of it. algorithms enjoy linear convergence guarantees and demonstrate good robustness to distribution shifts in practice over a range of application domains. I will also sketch interesting open questions for future research explorations.
This is based on joint work with K. Pillutla, R. Mehta, V. Roulet, Y. Laguel, J. Malick. Relevant papers include https://arxiv.org/abs/2201.00508, https://proceedings.mlr.press/v206/mehta23b.html. Relevant software packages include https://github.com/krishnap25/sqwash and https://yassine-laguel.github.io/spqr.
Bio:
Zaid Harchaoui is a Professor in the Department of Statistics with an adjunct appointment in the Paul G. Allen School of Computer Science and Engineering, and a Senior Data Science Fellow in the eScience Institute, at the University of Washington. He is a co-founder of IFDS, the NSF-TRIPODS institute on the foundations of data science, and IFML, the NSF-AI Institute on the Foundations of Machine Learning. He received the doctoral degree from Telecom Paris, now part of the Institut Polytechnique de Paris. He previously held appointments at the Courant Institute of Mathematical Sciences at New York University and at INRIA – the French National Institute for Research in Digital Science and Technology. He is currently an action editor at the Journal of Machine Learning Research, and an associate editor at the Journal of the Royal Statistical Society and the IEEE Transactions on Pattern Analysis and Machine Intelligence.