(578a) The Melon Toolbox: Machine Learning Models for Optimization
Recently, we have shown that a reduced-space formulation is favorable for deterministic global optimization with data-driven models embedded [9,10]. We propagate McCormick relaxations [11,12] through the model equations which allows operating in the degrees of freedom. Thus, the branch-and-bound algorithms branches only on the degrees of freedom and the sizes of the resulting subproblems are significantly reduced. This leads to significant speedups compared to a conventional full-space formulation and other solvers. For instance, we observed speedup factors of over 300 for ANNs and over 700 for GPs.
We propose the âMachine Learning Models for Optimizationâ (MeLOn) toolbox. The MeLOn toolbox provides a model library that includes various data-driven models from the machine-learning community. It interfaces with the state-of-the-art training toolboxes in TensorFlow, scikit-learn, and Matlab. Further, the extension to other toolboxes is possible. MeLOn  allows the user to easily integrate trained data-driven models into optimization problems. The tool is coupled to our open-source global solver MAiNGO which can solve problems in the reduced-space formulation. Therein, we have implemented convex and concave envelopes of relevant functions (e.g., activation functions of ANNs or covariance functions of GPs). This further accelerates optimization. In addition, we can parse optimization problems to GAMS. In this talk we provide an overview about recent advances in global optimization with data-driven models embedded. We also introduce the MeLOn toolbox and highlight a few engineering case studies.
The toolbox is open-source available at https://git.rwth-aachen.de/avt.svt/public/MeLOn/
 Lee, J. H., Shin, J., & Realff, M. J. (2018). Machine learning: Overview of the recent progresses and implications for the process systems engineering field. Computers & Chemical Engineering, 114, 111-121.
 Cozad, A., Sahinidis, N. V., & Miller, D. C. (2014). Learning surrogate models for simulationâbased optimization. AIChE Journal, 60(6), 2211-2227.
 Quirante, N., Javaloyes, J., Ruiz-Femenia, R., & Caballero, J. A. (2015). Optimization of Chemical Processes Using Surrogate Models Based on a Kriging Interpolation. In Computer Aided Chemical Engineering (Vol. 37, pp. 179-184). Elsevier.
 Boukouvala, F., & Ierapetritou, M. G. (2012). Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Computers & Chemical Engineering, 36, 358-368.
 Bradford, E., Schweidtmann, A. M., Zhang, D., Jing, K., & del Rio-Chanona, E. A. (2018). Dynamic modeling and optimization of sustainable algal production with uncertainty using multivariate Gaussian processes. Computers & Chemical Engineering, 118, 143-158.
 Meireles, M., Almeida, P., Simoes, M.G.: A comprehensive review for industrial applicability of artificial neural networks. IEEE Transactions on Industrial Electronics 50(3), 585-601 (2003).
 Nascimento, C.A.O., Giudici, R., Guardani, R.: Neural network based approach for optimization of industrial chemical processes. Computers & Chemical Engineering 24(9-10), 2303-2314 (2000).
 Henao, C.A., Maravelias, C.T.: Surrogate-based superstructure optimization framework. AIChE Journal 57(5), 1216-1232 (2011).
 Schweidtmann, A. M., & Mitsos, A. (2019). Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications, 180(3), 925-948.
 Schweidtmann, A. M., Bongartz, D., Grothe, D., Kerkenhoff, T., Lin, X., Najman, J., & Mitsos, A. (2020). Deterministic global optimization with Gaussian processes embedded in a reduced variable space. In Preparation.
 Mitsos, A., Chachuat, B., & Barton, P. I. (2009). McCormick-based relaxations of algorithms. SIAM Journal on Optimization, 20(2), 573-601.
 McCormick, G. P. (1976). Computability of global solutions to factorable nonconvex programs: Part IâConvex underestimating problems. Mathematical programming, 10(1), 147-175.