(662g) Clarifying Trust of Materials Property Predictions Using Neural Networks with Distribution-Specific Uncertainty Quantification

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches, Fig. 1. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network (CGCNN) to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 (OC20) dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

Fig. 1. Advanced materials discovery strategies are enabled with uncertainty quantification. Screening, active learning, and transfer learning are enhanced by trustworthy estimates of predictive uncertainty, particularly in high-throughput applications where the size of the uncertainty estimate is used to infer model accuracy. The oracle refers to some trustworthy system that outputs the desired target, such as DFT-accurate materials properties.