(33g) High-Performance, Scalable Nonlinear Programming Algorithms with Pynumero and Pyomo

Numerical optimization is a proven tool for optimal design, operation, and decision making for chemical processes. Furthermore, as we work to improve economics by considering more of the enterprise under a single optimization, these large-scale problems can quickly grow in size and complexity, outstripping the capabilities of standard, off-the-shelf algorithms.

PyNumero is a python package integrated with Pyomo to support rapid development of novel, computationally performant, nonlinear analysis tools and optimization algorithms. High-performance numerical algorithms are typically developed in low-level, compiled programming languages (e.g., C, C++, FORTRAN). While these languages provide the necessary performance, their complexity significantly increases implementation effort and time, impeding the development, prototyping, and testing of new ideas. PyNumero aims to provide a high-level programming environment for building advanced computational algorithms without sacrifices in computational performance.

PyNumero makes this possible by building off of Numpy and SciPy and providing python-based interfaces to compiled C++ and FORTRAN routines commonly used for nonlinear optimization algorithms. This includes efficient libraries like the AMPL Solver Library (ASL) and linear solvers in the Harwell Subroutine Library (HSL). PyNumero also supports the efficient sparse linear algebra routines found in SciPy. Through these interfaces, all scale-dependent linear algebra operations are performed in compiled code, in a way that is transparent to the developer. Furthermore, PyNumero allows for rapid development of numerical algorithms through unique support of parallel and serial block matrix structures. When coupled with the explicit support for the “Message Passing Interface” through Mpi4Py, this supports, for example, rapid, parallel assembly and solution of block-structured KKT systems on shared memory and distributed memory architectures. Moreover, because of the interfaces with Pyomo, an algebraic modeling language, PyNumero supports development of decomposition algorithms that are aware of, and can directly interrogate, the structure of the underlying optimization model (Rodriguez, Laird, and Zavala, 2020).

In this presentation, we will describe the new interfaces available in the PyNumero package. As well, we will show the computational performance of several examples of nonlinear programming algorithms implemented in PyNumero, including standard sequential quadratic programming and interior-point approaches (Wächter & Biegler, 2006), and novel algorithms for ill-conditioned or singular systems (Molina Thierry, 2020). We demonstrate the high computational efficiency of these algorithms compared with their purely C++ counterparts. Furthermore, we will show the performance and scalability of parallel, decomposition-based algorithms written in PyNumero, including novel decomposition approaches for solution of nonlinear dynamic optimization problems (Word et al. 2014) formulated in Pyomo.DAE (Nicholson et al., 2018).

References:

1. Rodriguez, J. S., Laird, C. D., & Zavala, V. M. (2020). Scalable preconditioning of block-structured linear algebra systems using ADMM. Computers & Chemical Engineering, 133, 106478.
2. Molina Thierry, “Nonlinear Optimization based frameworks for Model Predictive Control, State-Estimation, Sensitivity Analysis, and Ill-posed Problems”, Ph.D. Thesis, Carnegie Mellon University Department of Chemical Engineering, Jan. 2020.
3. Wächter, A., & Biegler, L. T. (2006). On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming, 106(1), 25-57.
4. Word, D. P., Kang, J., Akesson, J., & Laird, C. D. (2014). Efficient parallel solution of large-scale nonlinear dynamic optimization problems. Computational Optimization and Applications, 59(3), 667-688.
5. Nicholson, B., Siirola, J. D., Watson, J. P., Zavala, V. M., & Biegler, L. T. (2018). pyomo. dae: A modeling and automatic discretization framework for optimization with differential and algebraic equations. Mathematical Programming Computation, 10(2), 187-223.