SILO: Uncertainty Quantification for Foundation Models under Data and Compute Constraints

Abstract

Foundation models are increasingly used in settings where labeled data are scarce and adaptation must be fast and lightweight. But scarce data must often do double duty: we need labeled examples both to adapt a foundation model to a new task and to evaluate how much we can trust its predictions. Reusing the same data for both purposes creates subtle statistical dependencies.
In this talk, I will explore how we can make better use of limited labeled data without giving up statistical reliability. I will introduce new methods for aggregating predictors and reusing calibration data, together with finite-sample guarantees that hold even when classical exchangeability arguments break down. Experiments in vision and language illustrate how, in data-scarce settings, carefully designed data reuse can make foundation models both more useful and more trustworthy.

Bio

Maja Waldron is an Assistant Professor of Statistics at the University of Wisconsin–Madison. Her research develops statistical methods for trustworthy and efficient AI, with a focus on uncertainty quantification, conformal prediction, deep probabilistic modeling, and generative models. Before joining UW–Madison, she was a research scientist at the Bosch Center for AI. She received her PhD from Columbia University, where she was advised by David Blei.