(363ab) Operator's Eye Gaze Analytics for Evaluating Usability of Human Machine Interfaces in Process Control Applications

In a typical process industry, Human Machine Interfaces (HMIs) serve a crucial role in providing all aspects of communication and interaction of operators with processes, vital for process safety, quality, and efficiency (Ma et al., 2018). Human operators handle vast volumes of data and details of process variables, control systems, and set points. This necessitates obtaining data from the Human-Machine Interface (HMI), segregating it, and formulating the decision-making algorithms. Good interface design enables operators to accomplish their duties efficiently and effectively with minimal errors. Statistics reveal that about 70% of industrial accidents are due to human error (Mannan, 2013). Although, the industries started rely on automation, which minimizes the role of operators in the control rooms. However, when the automation underperforms, human factor becomes critical to deal with accidents. Surprisingly, the complex automation strategies in modern plants have made it harder for operators to comprehend the process due to information overload and sophisticated HMI that are above their capacity (Nazir et al., 2014). As a result, it is critical to evaluate usability of HMIs such that automation and operators work together.

An efficient interface aims to reduce the cognitive workload of operators and improve their performance. A significant amount of research has been done to evaluate the HMI in terms of operator response. For example, Howie and Vicente (1996) proposed to use operator performance measures such as reaction time, action transition graphs, the proportion of early actions, and process trajectory to evaluate the usability HMI. Lee et al. (2017) compared two different HMIs of a power plant control room based on operator performance. The measures used were reaction time (time to identify correct alarm and control action), efficiency (total time on task), and satisfaction (evaluated using system usability scale). Interface with better performance results in lesser reaction time and higher rating on system usability scale. These measures, however, ignore the cognitive abilities of operators (Ikuma et al., 2014). Cognitive abilities include operators’ information processing, reasoning with process state, and decision-making processes.

In recent years, physiological sensors such as eye-tracking, electroencephalography, and galvanic skin response have emerged to provide reliable insights into human cognition (Srinivasan et al., 2019). Among these, eye-tracking has emerged as a non-intrusive, non-invasive tool for measuring operators' trace of attention allocation on the HMI. A typical eye movement consists of fixation and saccade. Fixation is the period of time during which the eye remains still and is used to process information. Saccades are movements that occur between fixations. Several studies have been done to evaluate the usability of HMI using eye-tracking. For instance, Ikuma et al. (2014) used eye-tracking to compare operators' performance in two different HMIs of a typical petrochemical industry. An efficient interface should result in lesser search time. Shi et al. (2021) reported that for effective HMIs, the fixation to saccade ratio was higher, owing to the lesser search time. Our previous eye-tracking research with control room operators suggested that it can reveal operators' diagnostic and decision-making strategies during process anomalies. Operators who successfully completed a disturbance rejection task had specific eye gaze patterns on HMI, different from those who failed to complete the task (Sharma et al., 2016). Bhavsar et al. (2017) reported that gaze entropy obtained from eye-tracking could distinguish expert and novice operators. In another study, Shahab et al. (2021) developed an association metric to evaluate how operators establish relationships among process variables and control actions. Our results also indicate that a poorly designed interface may divert the operator's attention to a wrong variable, captured using the association metric. This can help provide guidelines for placing process elements on the HMI. More recently, Shahab et al. (2022) developed Hidden Markov Model based methodology to capture operators’ mental models using eye-tracking. The study of mental models is very relevant to HMI evaluation as it reveals how operators perceive the information from the HMI.

In this work, we propose to evaluate industrial HMIs using the cognitive study of control room operators. The experimental testbed consisted of a simulated ethanol process plant. A typical distributed control system is used to control the process. The operators need to monitor the process and intervene during abnormalities. The information from the DCS is presented to the operator on a display comprising three screens. Human factor studies were performed wherein eight graduate students from Indian Institute of Technology Madras volunteered to play the role of the control room operator. Prior to the experiment, the participants were provided with an instruction manual and technical handout to make them aware of their role as process operators. After training, the participants started the experiment, which involved six different disturbance rejection tasks. During the experiment, we recorded the process data, alarm information and operator actions. An in-house built eye tracker capable of recording eye movement data for multiple screen arrangements was used to record operators' eye movement.

For the purpose of usability evaluation of the HMI, the eye gaze data from the three displays are visualized using Visual Eye-tracking Analytics (VETA), an eye-tracking analytics toolbox (Goodwin et al., 2022). Fixations and saccades projected on the HMI are used to get the operators' attention on the HMI. These fixations are weighted by their durations, demonstrating how much attention has been spent on a certain area of HMI. Furthermore, we obtain insights regarding fixation transitions or saccades that occur due to a transition between two related regions on the HMI. If the transitions are frequent but the saccades are longer, it signifies poor placement of related variables on the HMI. This decreases the efficiency of the HMI and increases the response time for taking a control action. Our findings reveal several such insights from eye-tracking and lay the foundation for a systematic approach to evaluate usability of HMIs. In the future, these can be used to develop user-centric HMIs for process industries and thus help minimize human error.

Keywords: Control room operator, Eye-tracking, Human-Machine Interface, Cognitive behavior,

Acknowledgement- This work is partially funded by American Express Lab for Data Analytics, Risk and Technology, Indian Institute of Technology Madras

References

1. Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2017). Quantifying situation awareness of control room operators using eye-gaze behavior. Computers & chemical engineering, 106, 191-201.
2. Goodwin, S., Prouzeau, A., Whitelock-Jones, R., Hurter, C., Lawrence, L., Afzal, U., & Dwyer, T. (2022). VETA: Visual eye-tracking analytics for the exploration of gaze patterns and behaviours. Visual Informatics.
3. Howie, D. E., & Vicente, K. J. (1996, October). Measuring the Impact of Ecological Interface Design on Operator Skill Acquisition. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 40, No. 19, pp. 972-976). Sage CA: Los Angeles, CA: SAGE Publications.
4. Ikuma, L. H., Harvey, C., Taylor, C. F., & Handal, C. (2014). A guide for assessing control room operator performance using speed and accuracy, perceived workload, situation awareness, and eye tracking. Journal of loss prevention in the process industries, 32, 454-465.
5. Lee, S. T., Kim, S. Y., & Gilmore, D. (2017, October). Human-in-the-loop evaluation of human-machine interface for power plant operators. In 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 34-39). IEEE.
6. Ma, Z., Yoshikawa, H., Nawaz, A., & Yang, M. (2018). A human–machine interaction design and evaluation method by combination of scenario simulation and knowledge base. Journal of Nuclear Science and Technology, 55(5), 516-529.
7. Mannan, S. (2013). Lees' Process Safety Essentials: Hazard Identification, Assessment and Control. Butterworth-Heinemann.
8. Nazir, S., Kluge, A., & Manca, D. (2014). Automation in process industry: Cure or curse? How can training improve operator’s performance. In Computer Aided Chemical Engineering(Vol. 33, pp. 889-894). Elsevier.
9. Shahab, M. A., Iqbal, M. U., Srinivasan, B., & Srinivasan, R. (2021b). Metrics for objectively assessing operator training using eye gaze patterns. Process Safety and Environmental Protection, 156, 508-520.
10. Shahab, M. A., Iqbal, M. U., Srinivasan, B., & Srinivasan, R. (2022). HMM-based models of control room operator's cognition during process abnormalities. 1. formalism and model identification. Journal of Loss Prevention in the Process Industries, 76, 104748.
11. Shahab, M. A., Srinivasan, B., & Srinivasan, R. (2021a). Evaluating Control Room Operator Training Outcomes Through Eye Gaze Augmented Multi-Scale Data. In Computer Aided Chemical Engineering(Vol. 50, pp. 1307-1312). Elsevier.
12. Sharma, C., Bhavsar, P., Srinivasan, B., & Srinivasan, R. (2016). Eye gaze movement studies of control room operators: A novel approach to improve process safety. Computers & Chemical Engineering, 85, 43-57.
13. Shi, C., Noah, B., Rothrock, L., & Kim, J. H. (2021). Potential benefits of eye tracking within process control monitoring tasks. Human Factors and Ergonomics in Manufacturing & Service Industries, 31(3), 316-326.
14. Srinivasan, R., Srinivasan, B., Iqbal, M. U., Nemet, A., & Kravanja, Z. (2019). Recent developments towards enhancing process safety: Inherent safety and cognitive engineering. Computers & Chemical Engineering, 128, 364-383.