Automated weighting of data in CALPHAD – a comparison between frequentist and Bayesian approaches
Paulson, Noah H., S. Zomorodpoosh, I. Roslyakova, Stan, Marius .
CALPHAD XLVIII, Singapore, June 2nd - June 7th 2019, (2019)
The weighting of experimental datasets in the CALculation of PHAse Diagrams (CALPHAD) approach is a critical step in assigning appropriate error budgets and obtaining reasonable predictions. Currently, the assessor assigns weights based on a number of criteria, including the apparent quality of the measurements, thermodynamic consistency, the behavior of similar systems, and expert intuition . This represents a significant effort and is a barrier to the accelerated development of CALPHAD models for new systems. While with current technology these manual procedures are irreplaceable, the burden may be significantly reduced through the introduction of automated weighting schemes. Within the past year, at least two such statistical methods have been independently developed, one frequentist  and the other Bayesian . In this work, we describe both techniques and compare their strengths and weaknesses. We then use extensive sets of measurements of the low temperature specific heat and enthalpy of Hf and Al to compare the weights and models resulting from the dual methodologies. Finally, we examine the correlations between the automatically determined weights of both methods and metrics commonly employed by CALPHAD experts to weight datasets. The frequentist approach to dataset weighting utilizes the prediction error of each dataset as measured via modified K-Fold Cross Validation (KFCV). The weight of each dataset is scaled by the datasets with the highest and lowest prediction errors, respectively. In the Bayesian approach, the dataset weights are included as hyperparameters in the inference. These hyperparameters rescale the assumed errors of each dataset in the likelihood definition. Normalization is then performed to compare these weights to those obtained through the frequentist approach. Figure 1 demonstrates the results of the two methods in the selection of weights and in the prediction of models for the specific heat of the low temperature phase of Hf. Notice that the ranking of measurements by their weights are similar between the two methods. The resulting model predictions are similar as well. References  H. Lukas, S. G. Fries, and B. Sundman, Computational thermodynamics: the CALPHAD method. Cambridge University Press, 2007.  S. Zomorodpoosh, I. Roslyakova, A. Obaied, R. Otis, B. Bocklund, Z.-K. Liu, "Statistical approach for automated weighting of data and outlier detection". to be submitted, 2019.  N. H. Paulson, E. Jennings, and M. Stan, “Bayesian strategies for uncertainty quantification of the thermodynamic properties of materials,” arXiv Prepr. arXiv1809.07365, 2018.