What It Is
Uncertainty estimation is the task of assigning a confidence score to a model’s prediction — not just saying “the answer is X” but “the answer is X, and I’m 87% confident.” A well-calibrated model’s confidence should match its actual accuracy.
Why It Matters
In high-stakes applications (medical records, financial documents, legal text), knowing when to trust a model matters as much as the model’s average accuracy. Overconfident wrong predictions are more dangerous than acknowledged uncertainty.
How It Works
Several approaches exist: (1) Token-level confidence uses the model’s own output probabilities. (2) Semantic entropy runs multiple samples and measures how much the meaning of the answers varies. (3) Ensemble disagreement runs multiple independent models and measures how much their outputs differ — disagreement signals uncertainty. The ensemble approach is particularly powerful because it doesn’t require access to model internals: you only need the output text.
Key Sources
Related Concepts
- ensemble-methods — ensemble disagreement is a primary uncertainty signal
- self-consistency — consistency across samples is an uncertainty proxy