Enleashed

5.8 Calibration

A model tells you:

There is an 80% chance it will rain tomorrow.

Tomorrow arrives.

It rains.

Was the prediction good?

Perhaps.

But surprisingly, one rainy day tells us almost nothing about whether that 80% prediction was actually accurate.

To find out, we need to ask a different question:

Of all the occasions when the model predicted an 80% chance of rain, how often did it actually rain?

If the answer is approximately 80% of the time, the model is well calibrated.

If it rains only 40% of the time, the model is systematically too confident.

If it rains 95% of the time, the model is systematically underconfident.

This idea is called calibration, and it is one of the most important concepts for understanding predictions expressed as probabilities.

Prediction is not always a single answer

Many machine-learning systems do not naturally produce statements such as:

Rain.

or:

No rain.

Instead, they estimate something closer to:

[ P(Y=1 \mid X)=0.8 ]

which we can read as:

Given the information (X), the model estimates an 80% probability that outcome (Y) will occur.

A separate decision can then turn that probability into an action.

For example:

[ p(\text{rain}) = 0.8 ]

might lead a weather application to display:

80% chance of rain

while another system might apply a threshold:

[ p(\text{rain}) > 0.5 ]

and simply classify the day as:

Rain

These are different things.

The probability contains information about uncertainty.

The classification compresses that information into a decision.

Calibration asks whether those probabilities deserve to be believed.

What does calibrated mean?

Imagine collecting 1,000 predictions for which a weather model predicted:

[ p(\text{rain}) \approx 0.7 ]

Suppose rain subsequently occurred on 703 of those occasions.

Then:

[ \frac{703}{1000}=0.703 ]

The observed frequency is extremely close to the predicted probability.

That is evidence of good calibration.

More generally, for predictions assigned probability (p), we would like:

[ P(Y=1 \mid \hat{p}=p) \approx p ]

where:

(Y) is the actual outcome,
(\hat{p}) is the model's predicted probability,
(p) is a particular predicted probability.

In plain English:

Events predicted to happen with probability (p) should happen approximately (p) proportion of the time.

So:

Predicted probability	Desired observed frequency
10%	~10%
25%	~25%
50%	~50%
75%	~75%
90%	~90%

Calibration therefore connects predicted uncertainty with observed reality.

You cannot judge a probability from one event

This creates a slightly strange property of probabilistic prediction.

Suppose a doctor estimates:

There is a 20% probability that this patient will experience a particular complication.

The patient subsequently experiences the complication.

Was the prediction wrong?

No.

An event with probability 20% is expected to happen sometimes.

Likewise, suppose the complication does not occur.

That does not prove the prediction was correct either.

A probability describes what we should expect across repeated comparable predictions, not what must happen in one particular case.

Consider a fair six-sided die.

The probability of rolling a six is:

[ P(6)=\frac{1}{6} ]

If you roll the die once and obtain a six, that does not suddenly mean:

[ P(6)=1 ]

The probability described uncertainty before the outcome was known.

This distinction is fundamental whenever we evaluate forecasts.

Confidence is not the same as correctness

Suppose two models classify images of cats and dogs.

For one image:

Model B

[ P(\text{cat})=0.65 ]

Both predict:

Cat.

And suppose the image really is a cat.

Both models therefore receive the same classification accuracy for this example.

But their predictions are not equivalent.

Model A is saying:

I am almost certain.

Model B is saying:

Cat seems more likely, but there is substantial uncertainty.

If Model A routinely assigns probabilities around 99% while being correct only 80% of the time, it is overconfident.

If Model B assigns probabilities around 65% and is correct approximately 65% of the time, it may be much better calibrated.

Accuracy alone cannot reveal this difference.

Overconfidence

Consider an AI system making 10,000 predictions.

Whenever it reports approximately:

[ 90% ]

confidence, it is actually correct only:

[ 70% ]

of the time.

The model is overconfident.

Its probabilities imply greater certainty than its real-world performance justifies.

This matters enormously when humans use probabilities to make decisions.

Imagine the prediction is:

90% probability that this transaction is fraudulent.

If the true frequency among transactions receiving that score is only 60%, downstream decisions based on the stated probability may be badly distorted.

The problem is not necessarily that the model cannot distinguish risky transactions from safe ones.

The problem is that the meaning of its probability is wrong.

Underconfidence

The opposite can also occur.

Suppose predictions assigned:

[ P(Y=1)=0.6 ]

actually occur:

[ 85% ]

of the time.

The model is underconfident.

It has learned useful information, but its probability estimates systematically understate how predictive that information actually is.

Again, this matters because another system may use those probabilities.

A hospital might allocate resources according to estimated patient risk.

An insurer might price risk.

An electricity system might schedule reserves according to the predicted probability of a shortage.

A transport system might respond to predicted congestion.

If probabilities are used as inputs to subsequent decisions, their numerical meaning matters.

Reliability diagrams

One way of examining calibration is with a reliability diagram, sometimes called a calibration plot.

Predictions are grouped into probability ranges or bins.

For example:

[ 0.0-0.1 ]

[ 0.1-0.2 ]

[ \ldots ]

[ 0.9-1.0 ]

For each group, we calculate:

the average predicted probability;
the proportion of events that actually occurred.

We then compare them.

A perfectly calibrated model would approximately follow:

[ y=x ]

So predictions averaging 20% would correspond to outcomes occurring around 20% of the time.

Predictions averaging 80% would correspond to outcomes occurring around 80% of the time.

The closer the observed frequencies are to the predicted probabilities, the better calibrated the model is.

Calibration and discrimination are different

There is another important distinction.

A model can be good at ranking risk while being badly calibrated.

Imagine two groups.

The model assigns:

Group A: 20% risk
Group B: 80% risk

Suppose the actual risks are:

Group A: 5%
Group B: 40%

The model has correctly recognised that Group B is much riskier than Group A.

It therefore contains useful discriminative information.

But its probability estimates are badly calibrated.

Conversely, a model could be calibrated overall while being poor at distinguishing individuals.

Suppose an event occurs in 10% of a population.

A model could simply predict:

[ P(Y=1)=0.10 ]

for everyone.

Across the population, approximately 10% of those people might experience the event.

The model could therefore appear perfectly calibrated.

But it tells us nothing about who is actually at greater risk.

So a useful probabilistic model may need both:

discrimination — distinguishing higher-risk cases from lower-risk cases;
calibration — ensuring predicted probabilities correspond to observed frequencies.

These are related, but they are not the same thing.

Calibration can change over time

Suppose a model predicting loan defaults was trained using historical data.

At the time of training:

[ P(\text{default}\mid\hat{p}=0.2)\approx0.2 ]

The model is well calibrated.

Then the economy changes.

Interest rates rise.

Unemployment increases.

Household finances deteriorate.

People receiving a predicted default probability of 20% might now default 35% of the time.

The model has not necessarily changed.

The world has.

Its calibration has deteriorated because the relationship between its inputs and outcomes has changed.

This connects calibration to a concept we will examine more closely later:

distribution shift.

A model that was calibrated yesterday is not guaranteed to remain calibrated tomorrow.

Calibration can differ between places

The same problem exists geographically.

Imagine a healthcare model deployed nationally.

For patients receiving a predicted risk of:

[ 30% ]

the observed outcome might be:

Region A: 18%
Region B: 31%
Region C: 47%

Nationally, the model might appear reasonably calibrated.

Locally, it may not be.

Differences could arise because of:

population characteristics,
healthcare access,
environmental conditions,
measurement systems,
institutional practices,
economic circumstances,
or differences in the underlying process being predicted.

This returns us to an idea running throughout this course:

Aggregation can hide important information.

Calibration should therefore sometimes be examined not merely globally, but across:

time,
geography,
demographic groups,
operating conditions,
and other relevant contexts.

Why calibration matters for decision-making

Suppose an autonomous system must choose whether to take some action.

It predicts:

[ P(\text{failure})=0.02 ]

Whether 2% is acceptable depends on the consequence of failure.

For a trivial recommendation system, perhaps it is.

For an aircraft control system, perhaps it isn't.

For a medical intervention, the decision might depend on the relative consequences of treatment and non-treatment.

The prediction alone does not determine the correct action.

But if the decision system is going to reason about risk, it needs the probability to mean something.

This gives us an important chain:

[ \text{data} \rightarrow \text{model} \rightarrow \text{probability} \rightarrow \text{decision} \rightarrow \text{action} ]

If the probability is systematically miscalibrated, errors propagate downstream.

A decision system may be perfectly rational given the probabilities it receives, while still making poor decisions because those probabilities do not accurately describe reality.

What about large language models?

Calibration becomes especially interesting when we reach modern generative AI.

A language model fundamentally operates with probabilities over possible tokens.

Given some context, it estimates something resembling:

[ P(\text{next token}\mid\text{context}) ]

But humans interacting with an AI system are usually interested in something much harder:

How likely is it that what the model is telling me is actually true?

Those are not the same probability.

A model may generate an answer fluently because the sequence of words is highly probable given its learned representations.

That does not automatically provide a calibrated estimate of the factual correctness of the answer.

This is one reason apparent confidence can be misleading.

A system can produce:

“The answer is definitely X.”

with perfect linguistic confidence while X is false.

The language of certainty is not itself a calibrated measure of epistemic certainty.

This distinction will become extremely important when we later examine hallucination, reasoning and trustworthy AI.

Can we recalibrate a model?

Sometimes a model is useful but its probabilities are systematically distorted.

Rather than rebuilding the entire model, we can sometimes learn a transformation:

[ \hat{p} \rightarrow \hat{p}_{calibrated} ]

using separate validation data.

The objective is to make predicted probabilities better correspond to observed frequencies.

Methods such as:

Platt scaling,
isotonic regression,
temperature scaling

can be used for different types of models.

The important conceptual point is not the particular algorithm.

It is that probability estimates themselves can be evaluated and corrected.

We are not merely asking:

Did the model choose the correct answer?

We are asking:

Does the model understand how uncertain its predictions really are?

The deeper lesson

Imagine two AI systems.

The first says:

I am 99% certain.

The second says:

I am 70% certain.

The first sounds more impressive.

But if the first system is correct only 75% of the time when claiming 99% certainty, while the second is correct approximately 70% of the time when claiming 70% certainty, the second system is giving us a more truthful representation of what it knows.

That can be enormously valuable.

Because intelligent decision-making does not require pretending uncertainty does not exist.

It requires representing uncertainty well enough to make sensible decisions in spite of it.

Calibration therefore gives us another important principle for the course:

A trustworthy prediction should communicate not only what the model expects, but how much confidence we should actually place in that expectation.

And that takes us another step away from asking merely:

“Was the prediction right?”

towards the much more useful question:

“Did the model know how uncertain it was?”