Recent years have witnessed significant progress in the design and application of optimization- and learning-based controllers that can deal with multivariable dynamics, constraints, and uncertainties that appear in the system and/or the environment. However, the design of such advanced controllers hinges on the selection of several tuning parameters that may strongly affect closed-loop performance and constraint satisfaction. Due to the black-box nature of the relationship between tuning parameters and general closed-loop performance measures, there has been a significant interest in automatic calibration (i.e., auto-tuning) of complex control structures using derivative-free optimization methods as the corresponding objective function is expensive to evaluate, potentially nonconvex and multi-modal, and whose derivatives either do not exist or cannot be determined. Bayesian optimization (BO) [1] has recently been explored as a promising method that can handle noisy and expensive cost functions with unknown structure in the context of controller tuning (e.g., [2-4]). The main notion of BO is to represent the objective using a Gaussian Process (GP), i.e., a probabilistic surrogate model, which is learned by querying new points based on some, so-called, acquisition function. Nevertheless, an open challenge when applying BO to auto-tuning is how to effectively deal with uncertainties in the closed-loop system that cannot be attributed to a lumped, relatively small-scale noise term. In this talk, we will present an adversarially robust BO (ARBO) [5,6] method that is particularly suited to auto-tuning problems with significant time-invariant uncertainties in an expensive system model used for closed-loop simulations. We formulate the controller auto-tuning problem in the form of a nested, minimax optimization problem that aims to find the robustly (worst-case) optimal tuning parameters. ARBO relies on a GP model that jointly describes the effect of the tuning parameters and uncertainties on the closed-loop performance. From this joint GP model, ARBO uses an alternating (upper/lower) confidence-bound procedure to simultaneously select the next candidate tuning and uncertainty realizations to be queried, implying only one expensive closed-loop simulation is needed at each iteration. The adversarial robustness is tackled by alternating between an optimistic prediction of the performance measure to select the next best set of tuning parameters and a pessimistic prediction of the performance measure to select the most likely worst-case uncertainty for the suggested best tuning parameters. Subsequently, the selection of the robust optimal point is based on the posterior distribution of the candidate models for the objective, given all queried points so far. We will demonstrate the value of the proposed ARBO method on a challenging auto-tuning problem in which a highly nonlinear bioreactor with several unknown parameters is controlled using a nonlinear model predictive controller with multiple constraint backoffs that must be tuned. We will compare the quality of the ARBO solution to a random search, as well as a naïve mean-based robust BO approach.
[1] Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & De Freitas, N. (2015). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148-175.
[2] Sorourifar, F., Makrygiorgos, G., Mesbah, A., & Paulson, J. A. (2021). A data-driven automatic tuning method for MPC under uncertainty using constrained Bayesian optimization. IFAC-PapersOnLine, 54(3), 243-250.
[3] Piga, D., Forgione, M., Formentin, S., & Bemporad, A. (2019). Performance-oriented model learning for data-driven MPC design. IEEE control Systems Letters, 3(3), 577-582.
[4] Paulson, J.A. and Mesbah, A., (2020). Data-driven scenario optimization for automated controller tuning with probabilistic performance guarantees. IEEE Control Systems Letters, 5(4), pp.1477-1482.
[5] Bogunovic, I., Scarlett, J., Jegelka, S., & Cevher, V. (2018). Adversarially robust optimization with Gaussian processes. Advances in neural information processing systems, Vol. 31
[6] Paulson, J. A., Makrygiorgos, G., & Mesbah, A. (2021). Adversarially Robust Bayesian Optimization for Efficient Autoâ€Tuning of Generic Control Structures under Uncertainty. AIChE Journal, e17591.
