(635h) Objective Assessment of Operator Training Using Correspondence Analysis of Physiological and Behavioral Measures

In a typical process industry, operators are responsible for monitoring the state of the plant and perform appropriate control actions during abnormal situations. The operator gathers the input from various sources (primarily human machine interface), interprets it based on their knowledge and expertise (using mental models of the process), makes a prognosis, and then reacts to the situation in the form of various actions (physical or verbal). Ironically, the complex automation and control techniques have made it challenging for the operators to develop an appropriate mental model of causal relationships related to plant behavior (Kluge et al., 2014). The likelihood of committing errors during abnormal situation is very high often because of the incorrect mental models of the process developed by the operators (Leveson and Stephanopoulos, 2014). Thus, it is imperative to assess the mental models of the process developed by the operator while they train, so as to ensure adequate preparedness.

Process industries have long resorted to the use of Operator Training Simulators (OTS) for imparting training to the operators. Several studies have been conducted to evaluate operator performance during these training sessions using OTS. Typically, these approaches measure the learning and performance in terms of outcomes (success/failure), response times, deviation of operator actions from predefined sequences (Patle et al., 2014), etc. Nevertheless, these studies do not explicitly focus on evolution of mental models of the process (operator's knowledge about the dynamics of the process) during the training sessions. This is an important aspect as it helps understand the operator’s mental models and, therefore, can help develop effective individual-centric training programs. In addition, there are hardly few approaches that focus on understanding the cognitive behavior of operators, crucial to enhancing operators' skills and abilities (Kluge et al., 2014).. Therefore, it is necessary to account for cognitive aspects to assess the learning level of operators (Das et al., 2017) which can help improve training programs.

Recent development in sensor technology has enabled the researchers to get insights into the physiological behavior of humans (Manca et al., (2014). More specifically, eye tracking, a non-intrusive and non-invasive technology, has enabled researchers to get insights into humans' thought process (Srinivasan et al., 2019). Our previous research has shown interesting insights into the operator cognitive behavior (Bhavsar et al., 2017; Sharma et al., 2016; Kodapully et al., 2016). Our studies indicated that eye gaze analysis could identify the operator strategy in diagnosis and execution tasks during an experiment (Kodapully et al., 2016). Sharma et al. (2016) reported that many operators, even though successful in dealing with the abnormalities, have eye gaze orientation like that of the unsuccessful operators, thus providing additional insights into the operator performance. Statistical studies revealed that gaze-based entropy measures could be used to understand the situational awareness of the operators while handling plant abnormalities (Bhavsar et al., 2017). A digraph-based approach was proposed by Das et al. 2017 to evaluate the mental models of the operators based on the gaze transitions among different areas of interest on HMI. However, these were limited to single task analysis and that the evolution of mental models during training was not addressed.

In this work, we adopted correspondence analysis, a multivariate statistical technique, to quantify the development of mental models of the process by the operator. Correspondence analysis provides the relation between two categorical variables, in our case, regions on HMI (called as areas of interest, AOIs) and the different abnormal scenarios pertaining to the disturbance in the plant. We used a simulated ethanol production plant as a testbed and conducted human subjects’ experiments to evaluate the proposed approach. Ten participants were involved in the study. Each participant carried out repetitive tasks (repetition of a scenario) in different trials. A typical trial consists of six different disturbance scenarios. Overall, these ten participants performed 81 trials for each scenario leading to a total of 486 tasks. During these tasks, we recorded the process data, alarm information, operator actions data, and eye gaze data (using Tobi TX 300 eye tracker). To get the relation between the AOIs and different disturbance scenarios, we calculated the angle between these two on the asymmetric plot obtained from the correspondence analysis. A small angle between a scenario and an AOI represents the strong association between that scenario and AOI. Further, the distance to the origin of an AOI on the asymmetric plot indicates its importance in all scenarios. We observed signatures of learning with repetition of trials. For a typical operator in the last trial, the angle between scenario (corresponding to a particular disturbance) and relevant AOIs is found to be 0.90˚, very close to zero compared to 57˚ in the first trial. This indicates that with repetitive tasks, the operator has learnt the root cause of the disturbance (coolant flow) and discovered the correct action to be taken in response to the disturbance. Further, in the last trial, the distance to the origin of trend panel is found to be 0.06 (compared to 1.72 in the first trial) implying that the operator has understood the importance of trend panel and is observing the response of his actions by monitoring the process variable trend in all scenarios of the trial. Statistical studies reveal that the proposed metrics (obtained from correspondence analysis) can quantify learning during operator training programs. In this paper, we will describe the proposed approach and report results from the human subject experiments conducted to validate it.

References

1. Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2017). Quantifying situation awareness of control room operators using eye-gaze behavior. Computers & chemical engineering, 106, 191-201.
2. Das, L., Srinivasan, B., & Srinivasan, R. (2017). Cognitive behavior based framework for operator learning: knowledge and capability assessment through eye tracking. In Computer Aided Chemical Engineering (Vol. 40, pp. 2977-2982). Elsevier.
3. Kluge, A., Nazir, S., & Manca, D. (2014). Advanced applications in process control and training needs of field and control room operators. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3-4), 121-136.
4. Kodappully, M., Srinivasan, B., & Srinivasan, R. (2016). Towards predicting human error: Eye gaze analysis for identification of cognitive steps performed by control room operators. Journal of Loss Prevention in the Process Industries, 42, 35-46.
5. Leveson, N.G. and Stephanopoulos, G. (2014), A system‐theoretic, control‐inspired view and approach to process safety. AIChE J., 60: 2-14.https://doi.org/10.1002/aic.14278
6. Manca, D., Nazir, S., Colombo, S., & Kluge, A. (2014). Procedure for automated assessment of industrial operators. Chemical Engineering Transactions, 36, 391-396.
7. Patle, D. S., Ahmad, Z., & Rangaiah, G. P. (2014). Operator training simulators in the chemical industry: review, issues, and future directions. Reviews in Chemical Engineering, 30(2), 199-216.
8. Puskás, J., Egedy, A., & Németh, S. (2018). Development of operator training simulator for isopropyl alcohol producing plant. Education for Chemical Engineers, 22, 35-43.
9. Sharma, C., Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2016). Eye gaze movement studies of control room operators: A novel approach to improve process safety. Computers & Chemical Engineering, 85, 43-57.
10. Srinivasan, R., Srinivasan, B., Iqbal, M. U., Nemet, A., & Kravanja, Z. (2019). Recent developments towards enhancing process safety: Inherent safety and cognitive engineering. Computers & Chemical Engineering, 128, 364-383.