(635h) Objective Assessment of Operator Training Using Correspondence Analysis of Physiological and Behavioral Measures
AIChE Annual Meeting
Thursday, November 11, 2021 - 5:15pm to 5:30pm
Process industries have long resorted to the use of Operator Training Simulators (OTS) for imparting training to the operators. Several studies have been conducted to evaluate operator performance during these training sessions using OTS. Typically, these approaches measure the learning and performance in terms of outcomes (success/failure), response times, deviation of operator actions from predefined sequences (Patle et al., 2014), etc. Nevertheless, these studies do not explicitly focus on evolution of mental models of the process (operator's knowledge about the dynamics of the process) during the training sessions. This is an important aspect as it helps understand the operatorâs mental models and, therefore, can help develop effective individual-centric training programs. In addition, there are hardly few approaches that focus on understanding the cognitive behavior of operators, crucial to enhancing operators' skills and abilities (Kluge et al., 2014).. Therefore, it is necessary to account for cognitive aspects to assess the learning level of operators (Das et al., 2017) which can help improve training programs.
Recent development in sensor technology has enabled the researchers to get insights into the physiological behavior of humans (Manca et al., (2014). More specifically, eye tracking, a non-intrusive and non-invasive technology, has enabled researchers to get insights into humans' thought process (Srinivasan et al., 2019). Our previous research has shown interesting insights into the operator cognitive behavior (Bhavsar et al., 2017; Sharma et al., 2016; Kodapully et al., 2016). Our studies indicated that eye gaze analysis could identify the operator strategy in diagnosis and execution tasks during an experiment (Kodapully et al., 2016). Sharma et al. (2016) reported that many operators, even though successful in dealing with the abnormalities, have eye gaze orientation like that of the unsuccessful operators, thus providing additional insights into the operator performance. Statistical studies revealed that gaze-based entropy measures could be used to understand the situational awareness of the operators while handling plant abnormalities (Bhavsar et al., 2017). A digraph-based approach was proposed by Das et al. 2017 to evaluate the mental models of the operators based on the gaze transitions among different areas of interest on HMI. However, these were limited to single task analysis and that the evolution of mental models during training was not addressed.
In this work, we adopted correspondence analysis, a multivariate statistical technique, to quantify the development of mental models of the process by the operator. Correspondence analysis provides the relation between two categorical variables, in our case, regions on HMI (called as areas of interest, AOIs) and the different abnormal scenarios pertaining to the disturbance in the plant. We used a simulated ethanol production plant as a testbed and conducted human subjectsâ experiments to evaluate the proposed approach. Ten participants were involved in the study. Each participant carried out repetitive tasks (repetition of a scenario) in different trials. A typical trial consists of six different disturbance scenarios. Overall, these ten participants performed 81 trials for each scenario leading to a total of 486 tasks. During these tasks, we recorded the process data, alarm information, operator actions data, and eye gaze data (using Tobi TX 300 eye tracker). To get the relation between the AOIs and different disturbance scenarios, we calculated the angle between these two on the asymmetric plot obtained from the correspondence analysis. A small angle between a scenario and an AOI represents the strong association between that scenario and AOI. Further, the distance to the origin of an AOI on the asymmetric plot indicates its importance in all scenarios. We observed signatures of learning with repetition of trials. For a typical operator in the last trial, the angle between scenario (corresponding to a particular disturbance) and relevant AOIs is found to be 0.90Ë, very close to zero compared to 57Ë in the first trial. This indicates that with repetitive tasks, the operator has learnt the root cause of the disturbance (coolant flow) and discovered the correct action to be taken in response to the disturbance. Further, in the last trial, the distance to the origin of trend panel is found to be 0.06 (compared to 1.72 in the first trial) implying that the operator has understood the importance of trend panel and is observing the response of his actions by monitoring the process variable trend in all scenarios of the trial. Statistical studies reveal that the proposed metrics (obtained from correspondence analysis) can quantify learning during operator training programs. In this paper, we will describe the proposed approach and report results from the human subject experiments conducted to validate it.
- Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2017). Quantifying situation awareness of control room operators using eye-gaze behavior. Computers & chemical engineering, 106, 191-201.
- Das, L., Srinivasan, B., & Srinivasan, R. (2017). Cognitive behavior based framework for operator learning: knowledge and capability assessment through eye tracking. In Computer Aided Chemical Engineering (Vol. 40, pp. 2977-2982). Elsevier.
- Kluge, A., Nazir, S., & Manca, D. (2014). Advanced applications in process control and training needs of field and control room operators. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3-4), 121-136.
- Kodappully, M., Srinivasan, B., & Srinivasan, R. (2016). Towards predicting human error: Eye gaze analysis for identification of cognitive steps performed by control room operators. Journal of Loss Prevention in the Process Industries, 42, 35-46.
- Leveson, N.G. and Stephanopoulos, G. (2014), A systemâtheoretic, controlâinspired view and approach to process safety. AIChE J., 60: 2-14.https://doi.org/10.1002/aic.14278
- Manca, D., Nazir, S., Colombo, S., & Kluge, A. (2014). Procedure for automated assessment of industrial operators. Chemical Engineering Transactions, 36, 391-396.
- Patle, D. S., Ahmad, Z., & Rangaiah, G. P. (2014). Operator training simulators in the chemical industry: review, issues, and future directions. Reviews in Chemical Engineering, 30(2), 199-216.
- Puskás, J., Egedy, A., & Németh, S. (2018). Development of operator training simulator for isopropyl alcohol producing plant. Education for Chemical Engineers, 22, 35-43.
- Sharma, C., Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2016). Eye gaze movement studies of control room operators: A novel approach to improve process safety. Computers & Chemical Engineering, 85, 43-57.
- Srinivasan, R., Srinivasan, B., Iqbal, M. U., Nemet, A., & Kravanja, Z. (2019). Recent developments towards enhancing process safety: Inherent safety and cognitive engineering. Computers & Chemical Engineering, 128, 364-383.
This paper has an Extended Abstract file available; you must purchase the conference proceedings to access it.
Do you already own this?
Log In for instructions on accessing this content.
|AIChE Pro Members||$150.00|
|AIChE Emeritus Members||$105.00|
|AIChE Graduate Student Members||Free|
|AIChE Undergraduate Student Members||Free|
|AIChE Explorer Members||$225.00|