(529g) Robust Optimization with Hybrid First-Principles Data-Driven Models

Hybrid models are composed of rigorous first-principles models (FPMs) governed by necessary/known mechanisms, and data-driven models (DDMs) that account for complicated/unknown phenomena due to system complexity or lack of adequate system knowledge [1]. Hybrid models may offer significant advantages over both pure FPMs and pure DDMs because they have the ability to capture unknown phenomena and simultaneously adhere to critical process mechanisms. Thus, they not only provide practical solutions for modeling partially unknown systems, but they also have the potential to significantly improve model prediction accuracy. Hybrid modeling approaches have a wide range of applications in systems engineering fields, such as improvements of process outputs [2] and controller performance [3,4], and integrated system-level designs for highly complex processes [5,6]. Of particular interest here, is that utilizing hybrid models may significantly improve the design and operation of systems that must satisfy strict requirements in the face of uncertainty, such as safety-critical systems [2].

In process systems engineering, many systems require strict guarantees of performance/safety. From a model-based approach, robust design is a method capable of providing rigorous guarantees of performance/safety of process systems under uncertainty [7,8]. Deterministic global optimization methods are required to guarantee worst-case realizations of uncertainty are accounted for as modeling equations and performance/safety constraints are nonconvex, in the general case. These problems are typically formulated as bilevel programs, which are a class of optimization problems whose feasible set is characterized by another optimization problem. As such, they are used to model the adversarial nature of uncertainty in the design and operations of process systems. The bilevel optimization problems are extremely challenging or even impossible to solve directly using existing approaches. In this work, we consider equivalent semi-infinite programming (SIP) formulations for robust optimization problems [9] and we present a novel approach to account for hybrid models under uncertainty within SIPs. As a result of this work, hybrid modeling approaches are extended to problems that formally account for uncertainty and therefore enable their use in robust design and operations under uncertainty.

In this work, we present the formulations and solution strategies for different types of robust optimization problems with embedded hybrid models. Particular attention is paid to the hybrid models that use artificial neural networks (ANNs) for their DDMs with novel activation functions that are of interest for deep ANNs. We discuss our recent development of convex/concave envelopes of the novel sigmoid-weighted linear unit (SiLU) and Gaussian error linear unit (GELU) activation functions that enable the calculation of rigorous bounds on ANN outputs over the input domain. This further enables us to combine them with FPMs for deterministic global optimization and therefore robust optimization problems.

Several examples are implemented to demonstrate applications of different hybrid models in robust optimization applications, with specific attention paid to dynamical systems. The problems are solved to epsilon-global optimality using the EAGO solver [10] in Julia v1.5.1 [11]. Specifically, we present a case study for dynamic polymerization in a batch reactor with respect to the verification of robust operation. We also present a robust design problem for a continuous nitrification process in wastewater treatment. Finally, we present an interesting study involving a process model plagued by numerical issues caused by function domain violations. This problem was addressed with a novel approach that incorporates validity constraints and replaces the problematic models encountering domain violations with an ANN, demonstrating the applicability of hybrid models to overcome numerical issues in complicated process systems models.

References

[1] O. Kahrs and W. Marquardt, “The validity domain of hybrid models and its application in process optimization," Chemical Engineering and Processing: Process Intensification, vol. 46, no. 11, pp. 1054-1066, 2007.

[2] S. Zendehboudi, N. Rezaei, and A. Lohi, “Applications of hybrid models in chemical, petroleum, and energy systems: A systematic review," Applied Energy, vol. 228, pp. 2539-2566, 2018.

[3] B. Lennartson, M. Tittus, B. Egardt, and S. Pettersson, “Hybrid systems in process control," IEEE Control Systems, vol. 16, no. 5, pp. 45-56, 1996.

[4] F. J. Doyle, C. A. Harrison, and T. J. Crowley, “Hybrid model-based approach to batch-to-batch control of particle size distribution in emulsion polymerization," Computers & Chemical Engineering, vol. 27, no. 8-9, pp. 1153-1163, 2003.

[5] D. S. Lee, C. O. Jeon, J. M. Park, and K. S. Chang, “Hybrid neural network modeling of a full-scale industrial wastewater treatment process," Biotechnology and Bioengineering, vol. 78, no. 6, pp. 670-682, 2002.

[6] D. C. Psichogios and L. H. Ungar, “A hybrid neural network-first principles approach to process modeling," AIChE Journal, vol. 38, no. 10, pp. 1499-1511, 1992.

[7] M. D. Stuber and P. I. Barton, “Robust simulation and design using semi-infinite programs with implicit functions," International Journal of Reliability and Safety, vol. 5, no. 3/4, p. 378, 2011.

[8] M. D. Stuber, A.Wechsung, A. Sundaramoorthy, and P. I. Barton, “Worst-case design of subsea production facilities using semi-infinite programming," AIChE Journal, vol. 60, no. 7, pp. 2513-2524, 2014.

[9] M. D. Stuber and P. I. Barton, “Semi-infinite optimization with implicit functions," Industrial & Engineering Chemistry Research, vol. 54, no. 1, pp. 307-317, 2014.

[10] M. E. Wilhelm and M. D. Stuber, “EAGO.jl: easy advanced global optimization in Julia," Optimization Methods and Software, pp. 1-26, 2020.

[11] J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah, “Julia: A fresh approach to numerical computing," SIAM Review, vol. 59, no. 1, pp. 65-98, 2017.