How Sure Should We Be? Understanding Confidence Calibration
EXPLAINER @jorgebscomm for @empowervmedia Photo: Anna Keibalo via Pexels · Pexels License A confident judgement may be accurate, mistaken or somewhere between the two. Calibration asks whether our degree of certainty is justified by how reliably our judgements turn out to be correct. A commentator confidently announces that a proposed policy will succeed. Another thinks it will fail but expresses considerable uncertainty. Both predictions sound different. Yet neither person's confidence tells us, by itself, who is more likely to be correct. To assess their judgement, we would need to know what each predicted, the evidence available at the time and what eventually happened. More importantly, we would need to look beyond a single prediction. This is where confidence calibration becomes useful. The question is not whether someone sounds sufficiently doubtful, but whether their expressed certainty corresponds to their a...
