(434g) Goal-Oriented Model Learning with High-Fidelity Simulations Using MPC-Embedded Bayesian Optimization

The development, scale-up, and commercialization of next-generation / emerging process systems has traditionally relied on many steps that can range from lab-scale experimentation to pilot scale demonstration and, finally, to large-scale deployment. Computer simulations are increasingly used for optimizing designs of both individual (modular) process unit operations and overall integrated systems. However, modern high-fidelity simulation models often cover a wide range of length and time scales (from molecular- to device- to system-scales), which poses a significant computational burden for real-time optimization and advanced control applications. For such applications, a fast and accurate dynamic reduced model (D-RM) must be constructed from the computationally expensive high-fidelity model [1], [2]. A large number of methods exist for constructing D-RMs that range from grey-box models (that make application-specific approximations to unknown or expensive relationships in the model) to black-box models (highly parametrized models that can in principle realize any input-output map) [3]–[5]. The multivariable D-RM learning procedure is a critical bottleneck in advanced control design for two main reasons:

1. A large number of parameters may be needed to fit the behavior of the high-fidelity model throughout the entire space of time-varying inputs;
2. The process of generating enough data to accurately estimate the parameters of the D-RM can be expensive and/or time-consuming.

These challenges elude to the fundamental tradeoff between the speed and performance of D-RMs, which is usually addressed with some iterative procedure that requires expert tuning and decision-making to derive practically useful D-RMs. Although model learning (or identification) is usually separated from control design, we can instead think of the learning process as a procedure to be designed for the desired control application. This is motivated by the fact that the best D-RM for a given control task/goal may not be the one that produces the smallest output prediction error, but the one that provides the best performance on the high-fidelity model when in closed loop with the associated model-based controller. We refer to this idea as “goal-oriented model learning” and has strong ties to the identification for control (I4C) rationale [6] that has been studied for fixed-order (PID) control of linear time-invariant systems and has been recently extended to linear model predictive control (MPC) [7].

In this work, we introduce a novel framework for the automated construction of goal-oriented D-RMs from high-fidelity dynamic models. We focus on nonlinear MPC (NMPC) as the underlying control policy due to its flexibility in terms of handling nonlinear models, control objectives, and constraints [8]. Instead of treating the control-relevant model in NMPC as fixed, we instead treat it as a design parameter that will be optimized from closed-loop simulation data. Since the NMPC law is implicitly defined as the solution to an optimal control problem, the overall problem is naturally posed as a challenging bilevel optimization. Bilevel optimization problems can be transformed into single level optimization problems using reformulations that replace lower-level subproblems with their KKT conditions, e.g., [9] or their explicit multiparametric solutions, e.g., [10]. However, the KKT reformulation does not work for non-convex sub-problems because the KKT conditions are necessary but not sufficient for optimality in these cases. In addition, explicit multiparametric solutions cannot easily be computed for NMPC and are known to have issues scaling to problems with a large number of parameters. Therefore, we propose to tackle this challenging problem with Bayesian optimization (BO), which is an established approach for global optimization of black-box functions [11], [12]. The key advantages of BO are that it avoids derivative calculations (can be non-smooth in this case due to changes in the active set of the NMPC law) and allows for direct trade-off between exploration and exploitation through proper choice of the acquisition function. The proposed framework is demonstrated on a benchmark semibatch reactor problem that exhibits significant nonlinearity. We show how a D-RM of lower complexity is able to be estimated using closed-loop data that is geared toward the control task at hand, as opposed to open-loop data generated throughout the full range of input-output behavior. Moreover, we show how the same methodology can be used for optimal design of other key NMPC parameters (e.g., prediction horizon and constraint backoffs) by treating them as additional design variables during the BO procedure.

References

[1] J. Ma, P. Mahapatra, S. E. Zitney, L. T. Biegler, and D. C. Miller, “D-RM Builder: A software tool for generating fast and accurate nonlinear dynamic reduced models from high-fidelity models,” Comput. Chem. Eng., vol. 94, pp. 60–74, Nov. 2016, doi: 10.1016/j.compchemeng.2016.07.021.

[2] E. N. Pistikopoulos, N. A. Diangelakis, R. Oberdieck, M. M. Papathanasiou, I. Nascu, and M. Sun, “PAROC - An integrated framework and software platform for the optimisation and advanced model-based control of process systems,” Chem. Eng. Sci., vol. 136, pp. 115–138, Nov. 2015, doi: 10.1016/j.ces.2015.02.030.

[3] A. Juditsky et al., “Nonlinear black-box models in system identification: Mathematical foundations,” Automatica, vol. 31, no. 12, pp. 1725–1750, Dec. 1995, doi: 10.1016/0005-1098(95)00119-1.

[4] J. Roll, A. Nazin, and L. Ljung, “Nonlinear system identification via direct weight optimization,” in Automatica, Mar. 2005, vol. 41, no. 3, pp. 475–490, doi: 10.1016/j.automatica.2004.11.010.

[5] R. Romijn, L. Özkan, S. Weiland, J. Ludlage, and W. Marquardt, “A grey-box modeling approach for the reduction of nonlinear systems,” J. Process Control, vol. 18, no. 9, pp. 906–914, Oct. 2008, doi: 10.1016/j.jprocont.2008.06.007.

[6] M. Gevers, “Identification for Control: From the Early Achievements to the Revival of Experiment Design,” Conf. Pap. Eur. J. Control, vol. 11, pp. 1–18, 2006, doi: 10.1109/CDC.2005.1582109.

[7] D. Piga, M. Forgione, S. Formentin, and A. Bemporad, “Performance-oriented model learning for data-driven mpc design,” IEEE Control Syst. Lett., vol. 3, no. 3, pp. 577–582, Jul. 2019, doi: 10.1109/LCSYS.2019.2913347.

[8] M. Diehl, H. G. Bock, J. P. Schlöder, R. Findeisen, Z. Nagy, and F. Allgöwer, “Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations,” J. Process Control, vol. 12, no. 4, pp. 577–585, Jun. 2002, doi: 10.1016/S0959-1524(01)00023-3.

[9] Y. Zhan and Q. P. Zheng, “A multistage decision-dependent stochastic bilevel programming approach for power generation investment expansion planning,” IISE Trans., vol. 50, no. 8, pp. 720–734, Aug. 2018, doi: 10.1080/24725854.2018.1442032.

[10] N. P. Faísca, V. Dua, B. Rustem, P. M. Saraiva, and E. N. Pistikopoulos, “Parametric global optimisation for bilevel programming,” in Journal of Global Optimization, Aug. 2007, vol. 38, no. 4, pp. 609–623, doi: 10.1007/s10898-006-9100-6.

[11] D. R. Jones, M. Schonlau, and W. J. Welch, “Efficient Global Optimization of Expensive Black-Box Functions,” J. Glob. Optim., vol. 13, no. 4, pp. 455–492, 1998, doi: 10.1023/A:1008306431147.

[12] M. Pelikan, K. Sastry, and D. E. Goldberg, “Scalability of the Bayesian optimization algorithm,” Int. J. Approx. Reason., vol. 31, no. 3, pp. 221–258, Nov. 2002, doi: 10.1016/S0888-613X(02)00095-6.