New Framework for Evaluating Epistemic Uncertainty in AI

This research introduces a novel approach to evaluating epistemic uncertainty in AI models, shifting focus from conventional proxy tasks like out-of-distribution (OOD) detection and active learning. The authors argue that current evaluation methods often do not align with the Bayes-optimal decision strategies for quantifying epistemic uncertainty. Instead, they propose assessing epistemic uncertainty based on its capacity to identify "regret," which represents the reducible error in predictions. By framing selective prediction as a constrained optimization problem involving coverage, expected risk, and regret, the paper theoretically proves that the optimal selector is a thresholded convex combination of ground-truth aleatoric and epistemic uncertainties. This theoretical unification highlights a critical flaw in recent uncertainty disentanglement literature: standard correlation metrics between learned uncertainty components do not necessarily predict their actual operational utility. Consequently, the researchers advocate for evaluating the achievable risk, regret, and coverage surface of an uncertainty decomposition as a more diagnostic measure of joint disentanglement and utility. Benchmarking standard methods on datasets with dense human annotations reveals that decision-theoretic rankings can significantly diverge from proxy-task rankings, even showing rank inversions between methods considered top-ranked by one criterion and bottom-ranked by another. This underscores the need for a more robust evaluation framework.