A Tutorial on Physics-Informed Bayesian Optimization for Multi-Scale Process Design and Operation

Modern nonlinear programming (NLP) solvers can efficiently solve very large-scale optimization problems whenever first/second derivatives can be exactly computed. However, the development and formulation of multi-scale process models for which derivative information is readily available remains a significant challenge in many real-world applications. A particularly challenging class of problems is when at least one component of the model is very expensive or time-consuming to evaluate including high-fidelity computer simulations (e.g., molecular dynamics or density functional theory calculations) or laboratory experiments (e.g., measurement of critical material, chemical, or biological properties). In such cases, one must resort to “black-box” optimization methods (as opposed to rigorous “white-box” NLP approaches) as they assume very little about the structure of the objective and/or the constraint functions. Bayesian optimization (BO) has been one of the most widely applied approaches for so-called “data-driven optimization” in recent years due to its data efficiency. BO relies on the construction of probabilistic surrogate models to represent the posterior distribution for the objective and/or constraints. By combining these posterior distributions with an acquisition function, BO selects the next point at which to evaluate the unknown functions in a way that tradeoffs the exploration of regions where the surrogate model is uncertain and exploitation of the model’s confidence in good (feasible and low-cost) locations.

Although BO has been empirically shown to perform very well in a variety of application domains in which the problem dimensions is relatively small (on the order of 5-10), its sample efficiency tends to decrease as problem size increases due to exponential growth in the size of the search space. In recent years, there has been significant interest in the development of improved BO strategies that can overcome this limitation by selectively exploiting problem structure when possible. We broadly refer to these strategies as physics-informed BO (PIBO) (also known as grey-box BO). In this talk, we provide an overview of three recent advances in PIBO that can deliver considerable gains in performance in the context of process systems applications including (i) composite functions (e.g., represent partial knowledge on the structure of mass/energy balances), (ii) multi-fidelity model representations (e.g., represent access to cheaper approximations that can help guide the overall search process), and (iii) minimax problems (e.g., represent worst-case perturbations in critical uncertainties). We also demonstrate the achievable performance gains by these PIBO methods in relevant next-generation applications such as integrated design and control of flexible building HVAC systems, experimental calibration of genome-scale bioreactor models, and robust auto-tuning of nonlinear model predictive controllers.