(543c) Improved Long-Short Term Memory Model for Dynamic and Multimodal Processes Based on K-Means Clustering: Application to an Industrial 2, 3-Bdo Distillation Process

Owing to advancements in machine learning algorithms and increased accessibility to data, data-driven modeling has gained popularity in many fields. Recently, recurrent neural networks, especially Long-short term memory (LSTM), are widely used to predict temporal data in various applications [1, 2, 3, 4]. In industrial processes, LSTM networks are commonly used for process modeling [5], process monitoring [6], fault detection [7], and dynamic control [8]. Many previous studies proposed in the literature have developed LSTM networks based on synthetic data from open-loop simulations of first-principle models. Though these networks are proven to successfully reproduce simulated process dynamics, their applicability to industrial processes is not guaranteed as their training data describes only limited characteristics of a real-world process. Generally speaking, industrial processes are dynamic and operate under multimode regimes. Therefore, in order to utilize LSTM for practical applications, it is important to train it using raw data collected from an industrial process under various operating conditions. However, accurately capturing the temporal evolution of a process under dynamical and multimodal conditions is difficult.

To handle this challenge, in this work, we developed a new LSTM network utilizing clustered features relevant to normal operating conditions. First, we collect raw data representing dynamic, multimodal, and normal operating conditions of a process. Next, a clustering model that classifies the data based on its operating conditions is identified by applying K-means clustering to the collected raw data. We use Silhouette method to select an optimal number of clusters for enhanced clustering accuracy. Through the identified clustering model, we obtain training data corresponding to normal operating conditions. This data is further used to develop an LSTM network, which is used for process prediction in real-time. Additionally, we execute feature selection and noise filtering techniques to further improve the performance of LSTM. Data from the bio 2,3-BDO distillation process of GS Caltex, South Korea is modified and utilized to formulate a hypothetical case study to demonstrate the proposed method. This distillation process is dynamic with unexpected disturbances and operates under multimodal conditions. In this case study, we predicted the bottom product temperature using the proposed method. The results obtained proved the superior performance and stability of the developed LSTM as compared to the LSTM identified using raw data (without clustering-based feature selection).

Literature cited:

1. Hochreiter, S., JA1 4 rgen Schmidhuber (1997).“Long Short-Term Memory”. Neural Computation, 9(8).
2. Kwon, H., Oh, K.C., Chung, Y.G., Cho, H. and Kim, J., 2020. Development of Machine Learning Model for Predicting Distillation Column Temperature. Applied Chemistry for Engineering, 31(5), pp.520-525.
3. Li, J., Deng, D., Zhao, J., Cai, D., Hu, W., Zhang, M. and Huang, Q., 2020. A novel hybrid short-term load forecasting method of smart grid using MLR and LSTM neural network. IEEE Transactions on Industrial Informatics, 17(4), pp.2443-2452.
4. Choi, Y., An, N., Hong, S., Cho, H., Lim, J., Han, I.S., Moon, I. and Kim, J., 2022. Time-series clustering approach for training data selection of a data-driven predictive model: application to an industrial bio 2, 3-butanediol distillation process. Computers & Chemical Engineering, p.107758.
5. Zhang, J., Tang, Q. and Liu, D., 2019. Research into the LSTM neural network-based crystal growth process model identification. IEEE Transactions on Semiconductor Manufacturing, 32(2), pp.220-225.
6. Kadlec, P., Grbić, R. and Gabrys, B., 2011. Review of adaptation mechanisms for data-driven soft sensors. Computers & chemical engineering, 35(1), pp.1-24.
7. Ji, C. and Sun, W., 2022. A review on data-driven process monitoring methods: Characterization and mining of industrial data. Processes, 10(2), p.335.
8. Wang, Y., 2017, May. A new concept using lstm neural networks for dynamic system identification. In 2017 American control conference (ACC)(pp. 5324-5329). IEEE.