(415a) Modeling Complex Nonlinear Systems Using Concatenated Static-Dynamic Neural Networks
AIChE Annual Meeting
Wednesday, November 10, 2021 - 8:00am to 8:15am
Classical backpropagation algorithms for solving static and dynamic neural networks use first order methods, but these methods may require significant tuning of hyper parameters, can suffer from slow convergence, and may lead to difficulty in convergence in presence of inequality constraints. On the other hand, while second order methods can address some of the issues mentioned above, they can be computationally expensive due to Hessian calculation, may be limited in terms of candidate architectures, and may only be used for parameter estimating of small or medium size networks without incurring excessive computational expense. Therefore, applying the second order methods for the entire nonlinear static-dynamic network can be computationally expensive. Furthermore, the most efficient algorithm for solving the static network can be different than the most efficient algorithm for solving the dynamic network. Therefore, a novel strategy is developed where the static and dynamic networks are solved independently by efficient algorithms for those respective networks while solving an outer layer optimization for estimating the connection weights between the static and dynamic networks. Several outer layer optimization algorithms are proposed for efficient solution of the concatenated network. The developed algorithms are flexible for incorporating flexible network architectures and can include inequality constraints.
Various architectures are proposed for both the static and dynamic networks. In addition, various neuronal models with several candidate basis functions are considered to develop flexible networks that offer tradeoffs between computational expense and accuracy for highly nonlinear systems.
The proposed algorithms are applied to modeling the widely used Van de Vusse reactor and pH neutralization reactors as well as a highly complex superheater system with spatio-temporal variation, where complex dynamics associated with reactive-diffusive processes leading to oxide scale formation in the superheater tube banks coupled with mass and heat transfer makes it a challenging system. It is observed the concatenated static-dynamic neural network results in superior performance compared to the existing conventional static or dynamic networks taken separately or linear dynamic-nonlinear static networks4. Computational expense and convergence performance of the proposed algorithms are found to be far superior compared to the first order-only and second order-only methods especially for the superheater system. Impact of various basis functions on the computational expense and accuracy of the algorithms are also evaluated. Overall, proposed algorithms show promise for solving large nonlinear dynamic network problems.
- Su, H.-T., Bhat, N., Minderman, P. A. & McAvoy, T. J. Integrating Neural Networks With First Principles Models for Dynamic Modeling. Dynamics and Control of Chemical Reactors, Distillation Columns and Batch Processes (IFAC, 1993). doi:10.1016/b978-0-08-041711-0.50054-4
- Chen, L., Hontoir, Y., Huang, D., Zhang, J. & Morris, A. J. Combining first principles with black-box techniques for reaction systems. Control Eng. Pract. 12, 819â826 (2004).
- Sentoni, G. B., Biegler, L. T., Guiver, J. B. & Zhao, H. T. State-Space Nonlinear Process Modeling: Identification and Universality AIChE J. 44, 2229â2239 (1998).
- Mahapatra, P., Ma, J. & Zitney, S. E. Nonlinear Model Predictive Control Using Decoupled A-B Net Formulation for Carbon Capture Systems - Comparison with Algorithmic Differentiation Approach. Proc. Am. Control Conf. 2018-June, 6439â6444 (2018).
- Zhao, H., Guiver, J., Neelakantan, R. & Biegler, L. T. A nonlinear industrial model predictive controller using integrated PLS and neural net state-space model. Control Eng. Pract. 9, 125â133 (2001).