(104b) Physics-Informed Machine Learning Surrogates with Optimization-Based Guarantees: Applications to AC Power Flow | AIChE

(104b) Physics-Informed Machine Learning Surrogates with Optimization-Based Guarantees: Applications to AC Power Flow

Authors 

Jalving, J., Sandia National Laboratories
Eydenberg, M., Sandia National Labs
Blakely, L., Sandia National Labs
Surrogate modeling techniques have seen considerable success across engineering disciplines. Such approaches can utilize expert domain knowledge and can train on diverse datasets to represent challenging problems with tractable computational models [1,2]. Within the greater data-driven paradigm, continued hardware and algorithmic advances have enabled new innovations in surrogate modeling methodologies [3,4] that have led to successful engineering applications in planning, design, and control using massive amounts of available operational and simulation data [5].

Neural network approaches have recently received interest within engineering applications [6,7] for their ability to make high-accuracy predictions while being amenable to training with large-scale datasets [8]. Typical problems in machine learning however, are often concerned with the predictive aspects of neural network models (e.g. speech and image recognition), making their advantages less clear in the context of optimization where they take on the form of algebraic surrogates. For instance, design and control problems are often concerned with enforcing physical constraints (which may be implicitly defined on the inputs and outputs of a neural network) which are difficult to capture using conventional neural network training approaches. Furthermore, training high-accuracy neural networks often requires many nonlinear activation functions, which consequently hinders their usefulness as tractable surrogates. Neural networks are also most often trained using average predictions (e.g. mean squared error) on their output as opposed to using worst-case predictions. In the context of optimization, such worst-case predictions can be exploited without including proper model constraints, which necessitates verification of neural network performance [9].

In this talk, we present a methodology to develop and verify physics-informed neural network surrogates with application to the AC power flow (ACPF) equations [10,11]. We describe a lagrangian-dual approach [12] to capture the physical ACPF equations during training and we show how sparsification techniques [13] can reduce the size of the neural network to facilitate its use as an optimization-based surrogate. Next, we describe a verification framework that attains global guarantees on the worst-case neural network prediction by (i) exploiting relaxations of the true physical ACPF equations [14] and (ii) encoding a piecewise linear formulation of ReLU activation functions [15,16]. Our results demonstrate that the physics-informed neural network achieves vastly improved worst-case gaurantees versus training purely with data, and that the produced surrogate model is tractable and scalable within an optimization context. We lastly discuss extensions that use the produced neural network surrogate to formulate challenging problems for multiperiod AC optimal power flow and unit commitment coupled with power dispatch.

References:

[1] A. Bhosekar and M. Ierapetritou, “Advances in surrogate based modeling , feasibility analysis , and optimization : A review,” Computers and Chemical Engineering, 2018
[2] A. Cozad, N. V Sahinidis, and D. C. Miller, “A combined first-principles and data-driven approach to model building,” Computers and Chemical Engineering, 2015
[3] A. Cozad, N. V Sahinidis, and D. C. Miller, “Learning Surrogate Models for Simulation-Based Optimization,” AIChE Journal, 2014
[4] D. A. C. Beck, J. M. Carothers, V. R. Subramanian, and J. Pfaendtner, “Data Science : Accelerating Innovation and Discovery in Chemical Engineering,” AIChE Journal, 2016
[5] E. N. Pistikopoulos et al., “Process systems engineering – The generation next ?,” Computers and Chemical Engineering, 2021
[6] A. M. Schweidtmann and A. Mitsos, “Deterministic Global Optimization with Artificial Neural Networks Embedded,” Journal of Optimization Theory and Applications, 2019
[7] J. Eason and S. Cremaschi, “Adaptive sequential sampling for surrogate model generation with artificial neural networks,” Computers and Chemical Engineering, 2014
[8] Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, 2015
[9] A. Venzke, and S. Chatzivasileiadis, “Verification of Neural Network Behaviour : Formal Guarantees for Power System Applications,” IEEE Transactions on Smart Grid, 2019
[10] M. B. Cain, R. P. O’Neill, and A. Castillo, “History of Optimal Power Flow and Formulations,” Federal Energy Regulatory Commission, 2012.
[11] G. S. Misyris, A. Venzke, and S. Chatzivasileiadis, “Physics-Informed Neural Networks for Power Systems,” IEEE Power & Energy Society General Meeting, 2020
[12] F. Fioretto, T. W. K. Mak, and P. Van Hentenryck, “Predicting AC Optimal Power Flows: Combining Deep Learning and Lagrangian Dual Methods,” Proceedings of the AAAI Conference on Artificial Intelligence, 2020.
[13] J. Frankle, M. Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks,” International Conference on Learning Representations, 2019.
[14] D. K. Molzahn and I. A. Hiskens, “A Survey of Relaxations and Approximations of the Power Flow Equations,” 2019
[15] B. Grimstad and H. Andersson, “ReLU networks as surrogate models in mixed-integer linear programs,” Computers and Chemical Engineering, 2019
[16] M. Fischetti and J. Jo, “Deep Neural Networks and Mixed Integer Linear Optimization,” Constraints, 2018