(33f) An Algorithmic Toolbox for Surrogate-Based Optimization of Mixed- Integer Nonlinear Problems

High-fidelity computer simulations are routinely used in chemical process systems engineering to provide accurate data on complex physical phenomena, and these simulations typically consist of a complicated set of algebraic, ordinary, and/or partial differential equations. Despite significant advances in equation-based optimization and mathematical programming, directly embedding high-fidelity simulation equations within optimization formulations often leads to intractable problems. A promising solution to overcome this challenge is surrogate-based optimization. Surrogate-based optimization has been extensively studied for nonlinear problems (NLPs), and several algorithms currently exist [1-3]. However, many chemical engineering problems contain both continuous and discrete variables (i.e., mixed-integer nonlinear problems, or MINLPs), and only a few works currently exist for surrogate-based optimization for MINLPs. Existing surrogate-based MINLP algorithms relax the integrality constraint to construct smooth surrogate models [4-6]; however, this approach could degrade the performance of the algorithm [7].

In this talk, we present an open-source algorithmic toolbox for surrogate-based optimization of mixed-integer problems (SBO-MINLP). Unlike existing work, our algorithm allows the construction of mixed-integer surrogates, which directly handle discrete variables without relaxing integrality constraints. Our toolbox specifically targets constrained black or gray-box problems with continuous and binary variables as well as inequality and equality constraints. We have previously presented a surrogate-based optimization framework for MINLPs, which consists of two main search steps: 1) the MINLP search, where the most promising binary solution is determined; 2) the NLP search, where the algorithm further refines the solution by performing a search with only respect to continuous variables. The algorithm has solved problems up to 30 continuous and 8 binary variables and outperforms the relaxed surrogate modeling approach. In this work, we will present extensions of our previous work both with respect to methodology and algorithmic implementation.

First, the presented implementation synergistically employs an efficient sampling and data-preprocessing technique, machine learning, and adaptive sampling via optimization to effectively find an optimal solution. A data-preprocessing technique – one hot encoding – allows the construction of mixed-integer surrogate models that handle binary variables directly. Several surrogate types are supported, such as Artificial Neural Network (ANN), Gaussian Process models (GP), and Support Vector Regression (SVR). Most importantly, the software allows the use of different activation functions for ANNs (e.g., hyperbolic tangent function, rectified linear unit) coupled with appropriate problem reformulations to facilitate optimization. It also enables the user to build additional surrogate-types of their choosing. Second, our software allows efficient solution search via the use of parallel computing. Several components of the SBO-MINLP algorithm, such as the sampling of expensive computer simulation and hyperparameter searching for surrogate fitting, can be performed in parallel. Finally, we present heuristics for MINLP-NLP decomposition and identification of promising discrete solutions that are explored during the NLP search stage. Coupled with parallel computing, these strategies lead to significant computational cost savings. The performance of the toolbox will be shown through a set of benchmark problems as well as case studies for process synthesis and coupled material-process design optimization for an adsorption system [7].

References

1. Kim, S.H. and F. Boukouvala, Machine learning-based surrogate modeling for data-driven optimization: a comparison of subset selection for regression techniques. Optimization Letters, 2019.
2. Boukouvala, F. and C.A. Floudas, ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems. Optimization Letters, 2017. 11(5): p. 895-913.
3. Cozad, A., N.V. Sahinidis, and D.C. Miller, Learning surrogate models for simulation-based optimization. AIChE Journal, 2014. 60(6): p. 2211-2227.
4. Müller, J., MISO: mixed-integer surrogate optimization framework. Optimization and Engineering, 2016. 17(1): p. 177-203.
5. Rashid, K., S. Ambani, and E. Cetinkaya, An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization. Engineering Optimization, 2013. 45(2): p. 185-206.
6. Holmström, K., N.-H. Quttineh, and M.M. Edvall, An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optimization and Engineering, 2008. 9(4): p. 311-339.
7. Kim, S.H. and F. Boukouvala, Surrogate-Based Optimization for Mixed-Integer Nonlinear Problems. Computers & Chemical Engineering, 2020.