(362x) Ensemble Learning for Fault Diagnosis in Chemical Processes: Fusion of Results through a K-Nearest Neighbors Algorithm

Process monitoring is crucial to ensure safe operational reliability and prevent industrial accidents. With the increase in complexity of chemical plants and the non-linear characteristics of chemical processes, quick and correct fault detection and diagnosis of chemical processes become a difficult task. Although the control loop can compensate for many disturbances during process operations, it cannot handle significant disruptions[1]. So to overcome these problems, human intervention is required. Human operators are mainly responsible for process monitoring and fault diagnosis and can be overwhelmed due to alarm floods and inadequate time for troubleshooting. This motivates the need to develop automated, accurate fault detection and diagnosis technologies. In this paper, we seek to develop a robust data-driven fault detection technique that utilizes the strengths of different multiple fault detection techniques.

Recently, data-driven process monitoring methods have gained popularity because of the abundance of historical data [2]. From the literature, it has been observed that different statistical and machine learning methods have different fault detection performances for the same problem. This suggests that each classifier has its own range of expertise and inadequacies. Hence a combination of multiple classifiers with a suitable decision fusion strategy will help in improving fault detection and overall fault recognition performance. Decision fusion methods can be broadly categorized into Utility-based and Evidence-based methods. Utility-based methods (e.g., Majority voting, Min, Max, Product) do not use any prior knowledge of the predictions, whereas evidence-based methods (e.g., Naive-Bayes, Dempster-Shafer, Decision Template, KNN based fusion) use prior knowledge of the strengths and weaknesses of the method. Ghosh [3] performed evidence-based fusion strategies like weighted voting, Bayesian, and Dempster–Shafer on the Tennessee Eastman case study. They demonstrated that fusion offers complete fault coverage and increases the overall fault recognition rate vis-a-vis single classifiers. The results obtained from the above-mentioned fusion strategies did not provide any confidence to plant operators to make certain decisions, as there was limited information to interpret the results. In this work, we propose using the K- Nearest Neighbors (KNN) algorithm as a decision fusion strategy that will provide an additional localized neighbors information near the test sample which will help in Human-interpretable explanations for Decision making.

KNN has been successfully used in many domains due to its simple, fast, non-parametric, and interpretable results. The algorithm's flexibility lies in handling complex data that are difficult to explain using both Linear and non-linear relationships. KNN algorithm uses an input vector, which extracts non-linear features from the K nearest samples in its localized neighborhood. The final classification is performed by identifying the most common class among the K Nearest Neighbors [4]. In our proposed approach, KNN as a fusion strategy uses the weights assigned to a class by each classifier to make a classification decision. Assume that there are L different classifiers. For a sample X_tthe output from the i^thclassifier is given by F_i ( X_t ) = ( p_i,1, p_i,2 ,..., p_i,c ), where p_i,j is the degree of "support" given by a classifier F_i to c different classes. At the training stage, the degree of support obtained from the L classifiers are combined in a horizontal stacking manner to form a single vector V = [ F₁ ( X_t ), F₂ ( X_t ),...., F_L( X_t ) ] € R^D ( D = L x c ). For 'n' training samples, we will have 'n' vectors in D dimensions. As KNN is a memory-based algorithm, it stores all these 'n' training vectors in its memory. In the testing stage, a new sample is fed to all the L classifiers, and the degree of support obtained from all classifiers is horizontally stacked to a single vector form. The received input vector is then passed to KNN, which compares it with the K closest training samples in its feature space. While various distance metrics could be used for KNN, we have opted to use cosine similarity as the distance metric since it enables magnitude-independent comparisons. The final classification is obtained by majority voting among the K Nearest Neighbors.

In this paper, we describe the proposed KNN based fusion method and illustrate it using the well-known Tennessee Eastman challenge process [5]. We apply the proposed method using three classifiers – Deep Neural Networks(DNN), Convolutional Neural Networks( CNN), and Random Forest (RF). In terms of average diagnosis delay, Deep Neural Network performs the best (30 samples), which is about 23 % faster compared to Random forest (39 samples), and CNN delay is in between these two methods (34 samples). The overall fault recognition rate is best for CNN (77 %) and least for DNN (71%); RF is in between these two methods (76%). Since there is a wide range in the overall recognition rate and diagnosis delay of the three classifiers, an appropriate fusion strategy will improve the overall performance by reducing the total misclassification error made by each classifier. Bayesian method, when used as a fusion strategy, provided marginal improvement in the overall recognition rate (72 %) and diagnosis delay (35 samples) w.r.t slowest method, i.e., RF. Our proposed KNN based fusion strategy showed a significant improvement in the overall fault recognition rate from 71% to 81% and a 32 % reduction in average fault diagnosis delay (27 samples).

References

[1] M. Mansouri and H. Mohamed-Faouzi, Data-Driven and Model-Based Methods for Fault Detection and Diagnosis. Elsevier, 2020. doi: 10.1016/C2018-0-04213-9.

[2] P. Kadlec, B. Gabrys, and S. Strandt, "Data-driven Soft Sensors in the process industry," Comput. Chem. Eng., vol. 33, no. 4, pp. 795–814, 2009, doi: https://doi.org/10.1016/j.compchemeng.2008.12.012.

[3] K. Ghosh, Y. S. Ng, and R. Srinivasan, "Evaluation of decision fusion strategies for effective collaboration among heterogeneous fault diagnostic methods," Comput. Chem. Eng., vol. 35, no. 2, pp. 342–355, 2011, doi: https://doi.org/10.1016/j.compchemeng.2010.05.004.

[4] P. Cunningham and S. J. Delany, "K-Nearest Neighbour Classifiers - A Tutorial," ACM Comput Surv, vol. 54, no. 6, Jul. 2021, doi: 10.1145/3459665.

[5] J. J. Downs and E. F. Vogel, "A plant-wide industrial process control problem," Comput. Chem. Eng., vol. 17, no. 3, pp. 245–255, Mar. 1993, doi: 10.1016/0098-1354(93)80018-I.