SILO: Bayesian Optimization Beyond the Black Box: Leveraging Computational Structure for Efficient and Scalable Decision-Making

Abstract

Bayesian optimization (BO) is a principled framework for optimizing expensive, noisy objective functions, but traditional BO treats the system as a black box and learns only through input-output queries. In many scientific and engineering settings, this assumption is unnecessarily restrictive; valuable computational structure is often available, even if the full objective remains unknown. In this talk, I will describe a unifying “beyond black-box” viewpoint that suggests we treat BO as a sequential decision-making problem under uncertainty and explicitly encode whatever structure we have into the probabilistic surrogate model and acquisition strategy to improve both sample efficiency and computational scalability.

I will focus on three recurring forms of structure and the algorithmic ideas that they enable. First, compositional networks of functions, where the objectives and constraints arise from compositions of unknown components coupled by a known computational graph (which commonly occur in machine learning pipelines, physics-based simulation models, and dynamical/causal systems). Here, the known graph can be leveraged to construct decision rules that can be straightforwardly evaluated using modern nonlinear programming methods equipped with automatic differentiation tools. Second, multiple correlated information sources, where one can query potentially cheaper approximations of a high-fidelity objective function (e.g., hierarchies of simulators, coarse-to-fine spatial or temporal discretizations, and/or reduced-order models). I will show how specially designed cost-aware acquisition functions can identify the optimal input-source pair that maximizes information gain per evaluation cost. Third, unknown low-dimensional structure, where only a small subset of features significantly impact the objectives or constraints. I will illustrate how sparsity-inducing Bayesian modeling can adaptively identify task-relevant subspaces on the fly, enabling data-efficient molecular discovery over large descriptor libraries, with a real-world application to the design of sustainable battery materials.

I will close with some practical lessons and open questions about robustness when the assumed structural form of the probabilistic model is misspecified, and about the computational limits of acquisition optimization.

Bio

Joel Paulson is the Gerald and Louise Battist Associate Professor in the Department of Chemical and Biological Engineering at the University of Wisconsin-Madison. Prior to joining UW-Madison, he was a faculty member in the William G. Lowrie Department of Chemical and Biomolecular Engineering at The Ohio State University (OSU) from 2019 to 2025, where he held the H.C. Slip “Slider” Professorship. He holds a B.S. degree (with Highest Honors) from the University of Texas at Austin, and M.S.CEP. and Ph.D. degrees from the Massachusetts Institute of Technology (MIT), all in Chemical Engineering. After graduating from MIT, he was a postdoctoral scholar at the University of California, Berkeley, working in the area of systems and control theory. His current research interests are mainly in the areas of Bayesian optimization, scientific machine learning, and model predictive control, with applications in chemistry, biology, materials science, and manufacturing. He is the recipient of several awards including the CAREER Award from the National Science Foundation, the 35 Under 35 Award from the American Institute of Chemical Engineers (AIChE), and the 2020 Best Application Paper Prize at the World Congress of the International Federation of Automatic Control (IFAC).