(372c) Process Design Automation Using Reinforcement Learning
AIChE Annual Meeting
Tuesday, November 12, 2019 - 3:30pm to 5:00pm
Under the recognition of these problems, several methodologies to overcome the issues have been suggested. They include obtaining derivatives from a surrogate model, or not using derivative at all by utilizing the evolutionary algorithm [3,4]. They use the pattern of solutions from the sampling and enable to obtain the solution efficiently although it may not be exact. All these methods are relatively slow and, in many cases, do not guarantee convergence to global solutions, and generally hybridization with existing algorithms are preferred.
Nonetheless, methodologies mentioned earlier are all focused on solving a given design problem. If a new design problem is considered, the formulation must be repeated again, and the knowledge experienced during the solution procedure cannot be used to solve other problems. For example, if the feed composition or the parameters in reaction kinetics change, they need to formulate the problem and solve again.
This study presents a novel methodology of using reinforcement learning for a machine to perform diverse process design tasks and to master process design itself. We first establish an environment defined only with unit operations. We then develop an algorithm which converts the representation of a given process from a direct graph to a canonically equivalent tensor. We configure neural networks that take a process tensor as an input and output the desired type and specifications of a unit in the form of a probability distribution. Agent performing process design repeats the sequence of observing process tensor in each step, performing an action from the neural networks, and getting the profit of the process as a reward from the simulation result.
In numerous off-line learning processes, we could make the machine to go through diverse process design tasks by changing the feed composition and the parameters in reaction kinetics. We were able to confirm that the machine can obtain reliable solution not only from the tasks for which it was trained but also from the similar tasks that were not given during the training phase. Furthermore, we found that the more the machine learns, the shorter the time it took to design a process.
 Chen, Q., & Grossmann, I. E. (2017). Recent Developments and Challenges in Optimization-Based Process Synthesis. Annual Review of Chemical and Biomolecular Engineering, 8(1), 249â283. https://doi.org/10.1146/annurev-chembioeng-080615-033546
 Zhou, T., Zhou, Y., & Sundmacher, K. (2017). A hybrid stochasticâdeterministic optimization approach for integrated solvent and process design. Chemical Engineering Science, 159, 207â216. https://doi.org/10.1016/j.ces.2016.03.01
 Boukouvala, F., Misener, R., & Floudas, C. A. (2016). Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO. European Journal of Operational Research, 252(3), 701â727. https://doi.org/10.1016/j.ejor.2015.12.018
 Neveux, T. (2018). Ab-initio process synthesis using evolutionary programming. Chemical Engineering Science, 185, 209â221. https://doi.org/10.1016/j.ces.2018.04.015