(476f) Digitalizing the Process Industries Via Surrogate Optimization

Most optimization problems in engineering can be formulated as expensive black box problems whose solutions are limited by the number of function evaluations. Frequently, engineers develop models that are differentiable, cheap to evaluate and resemble reality. Under some conditions, namely if these models have algebraic expressions, these can be automatically differentiated to extract derivative expressions. Alternatively, if the function evaluations are cheap to compute, the derivatives can be approximated using finite difference methods. This gradient information can be leveraged within the framework of derivative-based optimization solvers (e.g. Newton’s method, Gradient descent) and the solution can then be transferred to the real system. In the absence of gradient information or cheap-to-evaluate models one must resort to efficient optimization routines that rely only on function evaluations. Creating a model can be itself considered part of the expensive black box optimization process.

Zeroth-order, data-driven or derivative-free optimization (DFO) has a long history in chemical engineering. Direct and model-based, local and global, as well as deterministic and random derivative-free optimization algorithms and software implementations have been compared for deterministic and stochastic case studies [1, 2]. Applications range from multi-scale and multi-level optimization to chemical synthesis and design of dynamic experiments [3, 4, 5, 6]. Model-based methods have attracted a lot of attention in process engineering, given the discipline’s expertise in surrogate, meta- and reduced order modelling [3, 7]. As such, the local trust-region methods and the global Kriging interpolation models have been found to be particularly promising in process optimisation [8, 9, 10, 11]. Under which conditions the additional function evaluations of heuristic or systematic partition-based global search space exploration methods are worth the potential increase in solution quality often remains problem-specific. To the best of the authors’ knowledge, there is no systematic study that compares different DFO algorithms on the basis of convergence and constraint satisfaction on a broad range of chemical engineering applications with varying amounts of stochasticity.

In this work, we investigate how different derivative-free optimization (DFO) algorithms can address different instances of problems in process engineering. On the algorithms side, we benchmark both model-based and direct-search DFO algorithms, which operate under different philosophies, as well as the alternative paradigm of finite difference approximation methods. On the problems side, the comparisons are made on illustrative toy-problem examples, as well as five chemical engineering applications: model-based design of experiments, self-optimizing reaction systems, flowsheet optimization, real-time optimization, and controller tuning. Various challenges, including constraint satisfaction, uncertainty, problem dimension, and evaluation cost are considered. In addition to this, the practical aspects of the implementation of these various algorithms are discussed, ranging from the ease of finding good hyperparameters to the applicability of these methods to specific problem instances.

This work provides insights into the efficiency of data-driven solutions of optimization problems in the process systems domain in an effort to advance the digitalization of the chemical and process industries.

References:

1. Rios, L. M., & Sahinidis, N. V. (2012). Derivative-free optimization: A review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3), 1247-1293. doi:10.1007/s10898-012-9951-y
2. Amaran, S., Sahinidis, N. V., Sharda, B., & Bury, S. J. (2015). Simulation optimization: A review of algorithms and applications. Annals of Operations Research, 240(1), 351-380. doi:10.1007/s10479-015-2019-x
3. Biegler, L. T., Lang, Y., & Lin, W. (2014). Multi-scale optimization for process systems engineering. Computers & Chemical Engineering, 60, 17-30. doi:10.1016/j.compchemeng.2013.07.009
4. Zhao, F., Grossmann, I., Muñoz, S. G., & Stamatis, S. (2020). Flexibility index of black-box models with parameter uncertainty through derivative-free optimization. doi:10.22541/au.160245361.19862481/v1
5. Felton, K., Rittig, J., & Lapkin, A. (2020). Summit: Benchmarking machine learning methods for reaction optimisation. doi:10.26434/chemrxiv.12939806
6. Olofsson, S., Schultz, E.S., Mhamdi, A., Mitsos, A., Deisenroth, M., & Misener, R. (2021). Design of Dynamic Experiments for Black-Box Model Discrimination. ArXiv, abs/2102.03782.
7. Bhosekar, A., & Ierapetritou, M. (2018). Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Computers & Chemical Engineering, 108, 250-267. doi:10.1016/j.compchemeng.2017.09.017
8. Eason, J. P., & Biegler, L. T. (2018). Advanced trust region optimization strategies for glass box/ black box models. AIChE Journal, 64(11), 3934-3943. doi:10.1002/aic.16364
9. Boukouvala, F., & Ierapetritou, M. G. (2014). Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE Journal, 60(7), 2462-2474. doi:10.1002/aic.14442
10. Boukouvala, F., & Floudas, C. A. (2016). ARGONAUT: Algorithms for global optimization of CONSTRAINED grey-box computational problems. Optimization Letters, 11(5), 895-913. doi:10.1007/s11590-016-1028-2
11. Del Rio-Chanona, E. A., Petsagkourakis, P., Bradford, E., Graciano, J. E., & Chachuat, B. (2021). Real-time optimization MEETS Bayesian optimization AND Derivative-free optimization: A tale of Modifier adaptation. Computers & Chemical Engineering, 147, 107249. doi:10.1016/j.compchemeng.2021.107249