(504g) The Nature of Fault-Space Classification By Neural Networks

Neural networks have emerged as a widely adopted tool used for process monitoring, fault diagnosis, and control. However, a major drawback is their inability to provide mechanistic explanations and insights about their decisions. In this paper, we report on a systematic approach to understand how a neural network performs classification by characterizing the structure of its feature space. Using carefully chosen model problems, we probe the structure of the feature space under various controlled conditions to elicit key insights. We show, for example, under what conditions adversarial attacks might succeed. We propose techniques to couple a neural network's learning with input modeling to make the system more robust in a classification task.