Variance and uncertainty

Expected value tells us where the centre of a distribution lies.

But it does not tell us how widely possible outcomes are spread around that centre.

That matters because two situations can have the same expected value and completely different levels of uncertainty.

Suppose two electricity-demand forecasts both have:

expected demand = 40 GW

But:

Forecast A

Demand is almost always between:

39 and 41 GW

Forecast B

Demand could plausibly lie between:

25 and 55 GW

The expected value is identical.

The uncertainty is not.

To describe that difference, we need a way of measuring spread.

One of the most important measures is variance.

Variance tells us how widely values tend to spread around their expected value.

Why the centre is not enough

Imagine two journeys.

Journey A

Travel time is:

30 minutes almost every day

Journey B

Travel time is:

sometimes 10 minutes,
sometimes 50 minutes,
averaging 30 minutes overall.

Both have the same expected travel time:

30 minutes

But if you need to arrive at an airport on time, they are very different services.

The average answers:

What journey time should I expect on average?

The spread answers:

How much might the actual journey differ from that expectation?

For many decisions, both matter.

Deviation from the mean

Suppose a random variable has expected value:

and we observe:

The observation lies:

2 units above the mean.

Its deviation is:

+2.

If we observe:

the deviation is:

-3.

So:

DEVIATION = VALUE - MEAN

Deviations tell us how far observations lie from the centre.

Why not simply average the deviations?

Suppose our values are:

8, 10, 12

The mean is:

10.

The deviations are:

-2, 0, +2

Their average is:

This always happens.

Positive and negative deviations cancel.

So simply averaging deviations does not tell us how spread out the data is.

We need another approach.

Squared deviations

Variance solves the cancellation problem by squaring the deviations.

For:

8, 10, 12

the deviations from the mean are:

-2, 0, +2

Square them:

4, 0, 4

Now they cannot cancel.

The variance is based on the average of these squared deviations.

Conceptually:

VARIANCE = average squared distance from the mean

Variance of a random variable

For a random variable X, variance is commonly written:

Var(X)

or:

σ²

The intuition matters more than the formula at this stage:

Low variance means outcomes are concentrated near the expected value.

High variance means outcomes are more widely spread.

Same mean, different variance

Consider two distributions.

Distribution A

Possible outcomes:

9, 10, 11

with most probability around 10.

Distribution B

Possible outcomes:

0, 10, 20

with substantial probability at the extremes.

Both could have:

E[X] = 10

But Distribution B has much greater variance.

This distinction is crucial whenever the consequences of being far from the average matter.

Standard deviation

Variance has one slightly awkward property.

Because deviations are squared, the units are squared too.

If electricity demand is measured in:

then variance is measured in:

GW².

That can be difficult to interpret.

So we often take the square root of variance.

This gives the standard deviation.

Standard deviation is commonly written:

σ.

It has the same units as the original variable.

So if:

demand is measured in GW

then:

standard deviation is also measured in GW.

Interpreting standard deviation

Suppose forecast demand has:

mean = 40 GW

and:

standard deviation = 1 GW.

That suggests the distribution is relatively tightly concentrated around 40 GW.

Now suppose:

standard deviation = 8 GW.

The central forecast is still 40 GW.

But possible outcomes are much more dispersed.

Standard deviation gives us an intuitive measure of how far outcomes typically lie from the centre.

Variance is not exactly the same as uncertainty

Variance measures one particular aspect of uncertainty:

spread around the mean.

But uncertainty is broader.

We may also be uncertain because:

the model is wrong,
the distribution may shift,
important variables are missing,
the data is limited,
we do not know which model is appropriate.

So:

Variance is a measure of dispersion within a probability model. Uncertainty includes uncertainty about the model itself.

This distinction will become increasingly important.

Aleatoric uncertainty

One useful distinction is between:

aleatoric uncertainty

and:

epistemic uncertainty.

Aleatoric uncertainty comes from variability that is inherent in the process.

For example, imagine customer arrivals.

Even if we know:

day,
time,
weather,
promotions,

we may still not know exactly who will enter the shop in the next ten minutes.

There is genuine variability in behaviour.

That remaining randomness is often described as aleatoric uncertainty.

More data may not remove aleatoric uncertainty

Suppose we observe customer behaviour for:

20 years.

We may estimate the arrival distribution extremely well.

But we still cannot predict with certainty exactly which customers will arrive tomorrow at 14:03.

Some variability remains.

More data can improve our estimate of the distribution.

It does not necessarily make the underlying process deterministic.

Epistemic uncertainty

Epistemic uncertainty arises because we do not know enough.

Perhaps:

we have little training data,
the sensor network is sparse,
model parameters are poorly estimated,
we have never encountered this situation before.

This type of uncertainty may be reducible through additional information.

For example:

more observations

may improve our knowledge.

A simple contrast

Suppose we want to predict tomorrow's temperature.

Aleatoric component

Weather itself is complex and variable.

Even with excellent information, some uncertainty remains.

Epistemic component

Perhaps one weather station is offline.

We therefore know less about the present atmospheric state.

Restoring the sensor may reduce this part of the uncertainty.

This distinction helps us ask:

Would collecting more information actually help?

Uncertainty can come from the data

Suppose a model is trained using only:

100 observations.

Its estimate of a relationship may be uncertain.

Train the same model using:

1,000,000 relevant observations.

Our uncertainty about the relationship may shrink.

This is uncertainty caused by limited evidence.

Uncertainty can come from the model

Suppose two models fit historical data equally well.

Model A predicts:

future demand = 40 GW

Model B predicts:

future demand = 47 GW

Which model is correct?

Perhaps we do not know.

That disagreement is another source of uncertainty.

This is model uncertainty.

Uncertainty can come from the state

Suppose a robot has noisy sensors.

It is uncertain whether it is located at:

Position A

or:

Position B.

Even if its dynamics model is perfect, uncertainty about the current state creates uncertainty about future predictions.

So future uncertainty may come from:

uncertain present state

plus:

uncertain future disturbances.

Uncertainty propagates

Suppose:

current position is uncertain.

Then we predict:

position in 1 second.

That prediction inherits the uncertainty.

Then:

position in 2 seconds

inherits uncertainty from both:

the previous uncertainty,
new disturbances.

So uncertainty often grows as we predict further into the future.

Conceptually:

UNCERTAIN PRESENT

↓

PREDICTION

↓

MORE UNCERTAIN NEAR FUTURE

↓

PREDICTION

↓

EVEN MORE UNCERTAIN DISTANT FUTURE

Variance can grow through time

Imagine a vehicle travelling at approximately:

20 m/s.

Suppose its speed is slightly uncertain.

After:

1 second

the possible positions remain relatively close together.

After:

60 seconds

small differences in speed can produce much larger differences in position.

The variance of the predicted position may therefore increase with the forecast horizon.

This is common in dynamic systems.

New observations can reduce uncertainty

Now suppose GPS provides a reliable position measurement.

The range of plausible positions narrows.

So:

PREDICT

→ uncertainty grows

then:

OBSERVE

→ uncertainty shrinks.

This cycle is fundamental to state estimation.

Prediction and correction

A state-estimation system often alternates between:

PREDICTION

and:

CORRECTION.

First:

Given the previous state and what we know about the system, where should we be now?

Then:

Given the new measurement, how should we update that estimate?

The result is:

estimated state + uncertainty.

Methods such as the Kalman filter formalise this idea.

Covariance

Variance describes uncertainty in one variable.

But systems often contain many uncertain variables.

Suppose a robot is uncertain about:

horizontal position,
vertical position.

These uncertainties may be related.

Perhaps if the robot is further east than expected, it is also likely to be further north.

To represent this relationship, we use covariance.

Covariance tells us how two uncertain variables tend to move together.

A covariance matrix

For a state vector such as:

x = [position, velocity]

we may want to represent uncertainty in:

position,
velocity,
the relationship between them.

This can be stored in a covariance matrix.

You do not need the matrix mathematics yet.

The conceptual idea is:

An uncertain state requires information about both how uncertain each variable is and how those uncertainties are related.

This becomes very important in robotics and control.

Correlated uncertainty

Suppose:

temperature forecast error

and:

electricity-demand forecast error

are related.

If temperature is colder than expected, demand may also be higher than expected.

Treating the two errors as independent could underestimate system risk.

The way uncertainties interact matters.

Variance of combined quantities

Suppose a system contains many uncertain demands.

If the demands are independent, some variation may cancel when aggregated.

If they are strongly correlated, they may rise and fall together.

So the variance of total demand depends not only on individual variances but also on correlations between users.

This is extremely important for shared-resource systems.

Diversification

Suppose two independent resources each have uncertain output.

When one produces less, the other may still produce normally.

Combining them can reduce relative variability.

This is the idea behind diversification.

It appears in:

finance,
energy systems,
supply chains,
resource portfolios.

But diversification works less well when uncertainties are strongly correlated.

Weather creates correlated uncertainty

Suppose thousands of rooftop solar systems exist across one region.

Their individual outputs vary.

But a large storm may reduce output across almost all of them simultaneously.

So the systems are not independent.

Geographic diversity can help, because weather conditions in distant locations may be less correlated.

Again:

space matters to uncertainty.

Averaging can reduce variance

Suppose many independent noisy sensors measure the same quantity.

Each sensor has:

random measurement error.

Averaging the measurements can produce a more stable estimate.

Random errors partly cancel.

This is one reason multiple measurements can improve confidence.

But systematic bias does not disappear in the same way.

Variance and missing data

Suppose a sensor goes offline.

We estimate the missing value using a model.

Immediately after failure, uncertainty may be small.

As time passes without measurement:

variance increases.

When the sensor returns:

variance decreases.

This provides a quantitative way to represent the principle from the previous module:

Missing data does not necessarily stop the system. It should increase uncertainty.

Confidence intervals

Suppose we estimate an unknown population quantity.

Instead of reporting only:

estimated value = 10

we might report:

estimate = 10

with:

confidence interval = 9.5 to 10.5.

The interval communicates uncertainty around the estimate.

A narrow interval suggests greater precision.

A wide interval suggests less.

Confidence intervals are about estimates

A confidence interval often describes uncertainty about a parameter or estimated quantity.

For example:

What is the average electricity consumption of the population?

A prediction interval answers a different question:

What value might the next individual observation take?

The two should not be confused.

Prediction intervals are usually wider

Suppose we estimate:

average household consumption = 10 kWh/day

very precisely.

Perhaps our uncertainty about the population average is small.

But individual households may consume:

3 kWh,
8 kWh,
15 kWh,
30 kWh.

So the prediction interval for one household may be much wider than the confidence interval around the average.

This distinction separates:

uncertainty about the mean

from:

variability among outcomes.

More data can narrow confidence intervals

Suppose we estimate average demand from:

10 observations.

Our estimate may be uncertain.

With:

100,000 representative observations

we can often estimate the historical mean much more precisely.

The confidence interval becomes narrower.

This is an example of epistemic uncertainty being reduced through data.

But prediction intervals may remain wide

Even if we know the distribution very well, individual outcomes may remain variable.

Suppose years of data tell us that journey times genuinely vary between:

20 and 60 minutes.

More historical data may allow us to estimate that distribution precisely.

It does not necessarily make tomorrow's journey predictable to within one minute.

Again:

knowledge about uncertainty

is different from:

absence of uncertainty.

Calibration and uncertainty

Suppose a forecasting system produces:

90% prediction intervals.

If the model is well calibrated, around:

90% of realised outcomes

should fall inside those intervals over repeated comparable forecasts.

If only:

60%

fall inside, the intervals are too narrow.

The model is overconfident.

If:

100%

fall inside enormous intervals, the model may be too conservative to be useful.

Sharpness versus calibration

A useful probabilistic forecast should ideally be:

calibrated

and:

sharp.

Calibration means:

The stated probabilities match observed frequencies.

Sharpness means:

The forecast is as concentrated as the available evidence allows.

A forecast saying:

Demand will be somewhere between 0 and 1,000 GW.

may have excellent coverage.

It is also useless.

Good uncertainty estimates should be honest without being unnecessarily vague.

Overconfidence

A model is overconfident when its predictions are more certain than its actual accuracy justifies.

For example:

99% confidence

but the model is correct only:

80% of the time.

Overconfidence is dangerous because downstream decision-makers may assume the estimate is much more reliable than it is.

Underconfidence

A model can also be underconfident.

Suppose it says:

60% confidence

for predictions that are correct:

95% of the time.

This is safer in some situations, but it may lead to:

unnecessary caution,
excessive reserve,
additional tests,
wasted resources.

Good uncertainty estimation matters in both directions.

Variance affects decisions

Suppose two power generators offer the same expected output:

100 MW.

Generator A has very low output variance.

Generator B has highly variable output.

If the system needs firm capacity at a specific time, they may have different value.

Expected output alone does not capture reliability.

Variance changes the service each resource provides.

Reliability is fundamentally probabilistic

Suppose a service promises:

Resource available whenever needed.

In reality, no physical system has perfect reliability.

Machines fail.

Networks fail.

Demand varies.

Reliability therefore concerns probabilities such as:

What is the probability the service is available when required?

or:

What is the probability demand exceeds available capacity?

Uncertainty becomes part of service design.

Variance and safety margins

Suppose expected demand is:

100 units.

If standard deviation is:

1 unit,

capacity of:

105 units

may provide a large margin.

If standard deviation is:

30 units,

the same capacity may be inadequate.

The required safety margin depends on uncertainty.

The cost of uncertainty

Greater uncertainty can require:

extra capacity,
reserves,
inventory,
insurance,
redundancy,
flexible resources.

These protections cost money.

So uncertainty has economic consequences.

Reducing uncertainty through better:

data,
forecasting,
sensors

can therefore create real value.

But certainty itself can be expensive

Suppose improving a forecast reduces standard deviation from:

5 units

to:

4.9 units

but requires:

€10 million of additional infrastructure.

Perhaps that information is not worth the cost.

Again, the right question is:

How much does reducing uncertainty improve the decision?

Not:

How do we eliminate all uncertainty?

Risk aversion

Two decision-makers may respond differently to the same variance.

Suppose an investment offers:

expected return = €100

with substantial uncertainty.

A wealthy investor may accept it.

Someone who cannot afford a loss may prefer a lower but more certain return.

This is risk aversion.

The same probability distribution can therefore produce different decisions depending on the decision-maker.

Variance is not always bad

Higher variance is not inherently undesirable.

Suppose an investment has:

possible large gains,
limited losses.

A decision-maker may actively prefer higher variability.

Similarly, exploration in AI deliberately accepts uncertain outcomes to learn something new.

Variance describes uncertainty.

Whether that uncertainty is good or bad depends on objectives and consequences.

Upside and downside variance are different

Variance treats deviations above and below the mean symmetrically.

Suppose an investment produces unexpectedly:

high returns.

Variance counts that as deviation just as it counts unexpectedly low returns.

But a decision-maker may care much more about downside risk.

This is one limitation of variance.

Tail measures

For some decisions, we care specifically about:

probability of exceeding a threshold,
worst plausible outcomes,
extreme losses.

For example:

P(demand > capacity)

may be much more relevant than overall variance.

Similarly:

P(robot collides with obstacle)

may matter more than average position error.

Variance is useful, but it is not the only risk measure.

Threshold probabilities

Suppose demand has:

mean = 40 GW

and:

standard deviation = 3 GW.

The system may ask:

What is the probability that demand exceeds 45 GW?

That probability directly connects uncertainty to a capacity decision.

A distribution allows us to calculate probabilities around meaningful thresholds.

Value at Risk

Finance sometimes uses a measure called Value at Risk, or VaR.

Conceptually, it asks something like:

How bad might losses become at a chosen probability level?

For example:

What loss will we exceed only 5% of the time?

This focuses attention on the tail rather than average variability.

The concept is useful beyond finance, though it also has limitations.

Expected shortfall

We may also care about:

If we do enter the bad tail, how bad is it on average?

This leads to ideas such as expected shortfall.

The broader lesson is:

Different decisions require different summaries of uncertainty.

Variance is one.

Tail probability is another.

Worst-case analysis is another.

Robust decisions

Suppose we do not trust the exact probability distribution.

Instead of optimising for one assumed distribution, we might choose an action that performs reasonably well across many plausible scenarios.

This is robust decision-making.

The question becomes:

Which action still works if our uncertainty model is somewhat wrong?

This can be valuable in systems where model uncertainty is substantial.

Worst-case decisions

At the extreme, we might plan around the worst possible outcome.

This can be appropriate where:

safety is critical,
failure is catastrophic.

But it can also be extremely expensive.

Design every system for the absolute worst imaginable scenario and resources may be massively underused most of the time.

Service design therefore often balances:

expected performance

variance

tail risk

cost

reliability.

Uncertainty should influence actions

Suppose two autonomous vehicles estimate:

pedestrian distance = 20 m.

Vehicle A has uncertainty:

±0.2 m.

Vehicle B has uncertainty:

±12 m.

The expected distance is identical.

Their actions should probably not be.

Vehicle B may:

slow,
gather more information,
increase its safety margin.

An intelligent system responds to uncertainty, not merely to expected state.

Decision confidence

Sometimes the state can be uncertain while the decision is still obvious.

Suppose temperature is estimated somewhere between:

18°C and 20°C.

If heating should activate only below:

10°C,

we do not need the exact temperature.

The state is uncertain.

The decision is not.

This is important:

We need enough certainty to choose the action, not necessarily enough certainty to know the world perfectly.

Uncertainty near boundaries matters more

Now suppose the heating threshold is:

19°C.

Our estimate is:

18–20°C.

Suddenly uncertainty matters greatly.

Some plausible states imply:

heating ON.

Others imply:

heating OFF.

This is where additional information may be valuable.

Decision uncertainty

We can therefore distinguish:

state uncertainty

from:

decision uncertainty.

A system may be highly uncertain about an exact value but still confident about the correct action.

Or a small amount of state uncertainty near a critical threshold may make the decision very uncertain.

This is a powerful idea for intelligent service design.

Measure uncertainty relative to the task

Suppose GPS uncertainty is:

±5 metres.

For:

Which city am I in?

that uncertainty is irrelevant.

For:

Which motorway lane am I in?

it may be critical.

There is no universal level of "good enough" uncertainty.

It depends on what the system must do.

Uncertainty and AI hallucination

Suppose a language model has strong evidence for an answer.

Its internal distribution may be highly concentrated.

Now suppose the question concerns an obscure fact for which training evidence is weak.

The model may still generate a fluent answer.

If the system does not expose its uncertainty, both answers can appear equally authoritative.

This creates a serious interface problem:

Fluency can hide variance in the model's confidence.

AI should distinguish knowing from guessing

Ideally, an intelligent system should distinguish between:

I have strong evidence

I have a plausible inference

I have weak evidence

I do not know

These are different epistemic states.

If the interface collapses them into equally confident language, uncertainty has been hidden.

Ensembles

One way to estimate model uncertainty is to use several models.

Suppose:

Model A predicts 10,
Model B predicts 10.2,
Model C predicts 9.9.

They agree closely.

Now suppose:

Model A predicts 5,
Model B predicts 12,
Model C predicts 30.

The disagreement suggests substantial uncertainty.

Collections of models like this are called ensembles.

Disagreement can reveal uncertainty

This is not perfect.

Several models can all be wrong together.

But model disagreement can provide useful evidence about uncertainty.

If many plausible models produce very different answers, confidence should probably be lower.

This is conceptually similar to asking several experts.

Uncertainty and learning

Suppose a model encounters:

a familiar example.

Uncertainty is low.

Now it encounters:

an unusual example far from its training data.

Uncertainty rises.

This can guide learning.

The system may prioritise uncertain examples for:

human labelling,
further measurement,
exploration.

Uncertainty becomes an input to deciding what to learn next.

Active learning

Suppose an AI has one million unlabelled examples but can afford human labels for only:

1,000.

Instead of selecting randomly, it might choose examples where:

model uncertainty is highest.

The system uses uncertainty to allocate scarce labelling resources.

This is active learning.

Again, uncertainty connects directly to resource allocation.

Exploration in robotics

A robot may know one part of a room extremely well.

Another area remains uncertain.

If its objective includes improving its map, it may deliberately explore the uncertain region.

So:

UNCERTAINTY

↓

CHOOSE WHERE TO OBSERVE

↓

NEW DATA

↓

LOWER UNCERTAINTY

The machine actively manages its uncertainty.

Uncertainty becomes part of state

We can now strengthen our concept of state.

Instead of representing only:

estimated position = 10

a machine may internally represent:

estimated position = 10

plus:

uncertainty = small.

The state of the intelligent system therefore includes not only:

what it believes

but also:

how strongly it believes it.

This is essential in partially observable environments.

From point state to belief state

If the true state is uncertain, we can maintain a belief state.

Rather than:

x = one exact state

we have:

distribution over possible states.

That distribution has:

an expected value,
variance,
perhaps multiple peaks.

This idea is foundational to:

Bayesian filtering,
robotics,
partially observable decision processes.

The full intelligent loop

Our framework can now become:

WORLD

↓

NOISY + PARTIAL OBSERVATIONS

↓

ESTIMATED STATE

UNCERTAINTY

↓

DISTRIBUTION OVER FUTURE STATES

↓

EXPECTED OUTCOME + VARIANCE + TAIL RISK

↓

DECISION

↓

ACTION

↓

NEW WORLD

This is much richer than:

data → prediction.

Expected value and variance belong together

Expected value tells us:

Where is the distribution centred?

Variance tells us:

How widely is the distribution spread?

Together they provide a basic description:

CENTRE + SPREAD

But even those two numbers do not describe every distribution completely.

We may still care about:

skew,
multiple peaks,
tails,
correlations.

Uncertainty cannot always be compressed into one or two numbers.

Ask about spread as well as centre

Whenever someone gives you a forecast, ask:

What is the expected value?
What is the variance?
What is the standard deviation?
How wide is the prediction interval?
What causes the uncertainty?
Is it aleatoric or epistemic?
Would more data reduce it?
Does uncertainty grow with forecast horizon?
Are variables correlated?
Are there important tail risks?
How does the uncertainty affect the decision?
Is the model calibrated?
Does the model know when it is outside familiar territory?

These questions move us beyond point predictions.

The central idea

Expected value tells us where possible outcomes are centred.

Variance tells us how much they tend to spread around that centre.

But uncertainty is broader than variance.

It may arise from:

inherent randomness,
incomplete observations,
limited data,
uncertain models,
changing environments.

A useful intelligent system should therefore represent both:

what it thinks

and:

how uncertain it is.

The conceptual progression becomes:

OBSERVATION

↓

ESTIMATED STATE + UNCERTAINTY

↓

PROBABILITY DISTRIBUTION

↓

EXPECTED VALUE + VARIANCE + TAILS

↓

DECISION

And uncertainty should not simply disappear as information flows through the system.

Good intelligence does not turn uncertain inputs into falsely certain outputs. It carries uncertainty forward until the decision no longer depends on it — or until more information is required.

So far, we have treated probabilities mostly as standalone quantities.

But probabilities often change when we learn something new.

Knowing that it is raining changes the probability that someone is carrying an umbrella.

Knowing that a medical test is positive changes the probability that a patient has a disease.

In the next lesson, we will formalise this idea through conditional probability: how probabilities change when we know that something else is true.