Enleashed

A random variable tells us what quantity is uncertain.

A probability distribution tells us how that uncertainty is spread across the possible values.

Suppose:

X = tomorrow's electricity demand at 18:00

Knowing only the variable does not tell us very much.

We also want to know:

which values are plausible,
which are unlikely,
which are most likely,
how spread out the possibilities are,
whether there are extreme outcomes we should care about.

A probability distribution gives us that picture.

A distribution is a map of uncertainty across possible outcomes.

A distribution is more than one number

Suppose two forecasting models both predict:

expected demand = 40 GW.

At first glance, they look identical.

But imagine:

Forecast A

Most plausible outcomes lie between:

39 and 41 GW.

Forecast B

Plausible outcomes lie between:

30 and 50 GW.

The average is the same.

The uncertainty is completely different.

This is why a single point forecast can hide important information.

A simple discrete distribution

Suppose:

X = number of customers arriving in the next 10 minutes.

We estimate:

X	Probability
0	0.05
1	0.15
2	0.30
3	0.30
4	0.15
5	0.05

This is a probability distribution.

It tells us that:

2 and 3 customers are the most likely outcomes,
0 or 5 customers are possible but less likely.

The probabilities add up to:

The shape of the distribution matters

Imagine two distributions.

Distribution A

Most probability is concentrated near the centre.

Distribution B

Probability is spread across a much wider range.

Both may have the same average.

But Distribution B is more uncertain.

The shape of the distribution tells us something about the structure of the uncertainty.

Peaks

A distribution often has one or more regions where probability is concentrated.

These are its peaks.

Suppose tomorrow's temperature is most likely to be around:

18°C.

The distribution may peak near 18.

That means values close to 18 are more plausible than values far away.

The highest point is often associated with the mode.

The mode is the most probable or most common value, depending on the context.

A distribution can have more than one peak

Suppose electricity demand tomorrow depends strongly on whether a major industrial plant is operating.

If the plant runs:

high demand

If it does not:

lower demand

The distribution may contain two peaks.

One around:

35 GW

and another around:

45 GW.

This is called a multimodal distribution.

A single average around:

40 GW

might actually describe an outcome that is relatively unlikely.

The average can sit where little probability exists

This is an important point.

Suppose there is:

50% chance of 30 GW,
50% chance of 50 GW.

The expected value is:

40 GW.

But:

40 GW may never actually occur.

The average is mathematically meaningful.

It is not necessarily a likely outcome.

This is why we should not confuse:

expected value

with:

most likely value.

Distributions can be symmetric

A distribution is symmetric when its left and right sides are roughly mirror images.

For example, measurement errors around a true value are often modelled approximately symmetrically.

If the most likely value is:

then:

and:

might have similar probabilities.

Symmetry can simplify analysis.

But real-world distributions are not always symmetric.

Skewed distributions

A distribution is skewed when one side extends further than the other.

Income is a common example.

Many people may earn around moderate values, while a smaller number earn extremely high incomes.

This produces a long upper tail.

The distribution is right-skewed.

In such cases:

mean

and:

median

can differ substantially.

Why skew matters

Suppose average income is:

€70,000.

If the distribution is strongly right-skewed, most people may earn much less.

So knowing the distribution helps us interpret the average.

The same can happen with:

house prices,
waiting times,
insurance losses,
electricity prices,
financial returns.

Extreme values can pull averages away from typical experience.

Tails

The outer regions of a distribution are called its tails.

These contain less likely outcomes.

For example:

very low demand

or:

very high demand.

The tails may contain little probability.

But they can be extremely important.

A rare outcome can still have severe consequences.

Tail risk

Suppose expected electricity demand is:

40 GW.

Most of the time, demand lies between:

38 and 42 GW.

But there is a:

1% probability

of demand above:

50 GW.

That 1% may matter enormously if the system fails above 50 GW.

This is tail risk.

Rare outcomes can matter more than common ones when their consequences are large.

The centre is not the whole story

Many summaries focus on the centre of the distribution.

For example:

mean,
median,
mode.

These are useful.

But decision-making may depend more on:

spread,
asymmetry,
tails,
thresholds.

A distribution contains all of these.

Spread

The spread describes how far values tend to vary around the centre.

Suppose:

Model A

Values usually fall between:

9 and 11.

Model B

Values usually fall between:

0 and 20.

Both may be centred on:

10.

Model B has much greater spread.

Later we will use variance and standard deviation to quantify this.

Narrow distributions imply greater certainty

Suppose a weather forecast says:

Tomorrow's temperature is likely between 19°C and 21°C.

That is relatively precise.

Now compare:

Tomorrow's temperature is likely between 5°C and 30°C.

The central estimate may be the same.

But the second forecast expresses far greater uncertainty.

Narrow distributions generally imply more concentrated belief.

Wide distributions imply more uncertainty.

A distribution can change with new information

Suppose our initial forecast for tomorrow's demand is:

broad and uncertain.

Then we learn:

Tomorrow is a public holiday.

The distribution may shift downward.

Then we learn:

A severe cold snap is expected.

The distribution may shift upward again.

Probability distributions are not fixed forever.

They reflect what we know at a particular moment.

Conditional distributions

Suppose:

D = tomorrow's electricity demand

and:

T = temperature.

Instead of asking only:

P(D)

we may ask:

P(D | T = 0°C).

This means:

What is the distribution of demand given that temperature is 0°C?

The distribution may be very different from:

P(D | T = 20°C).

Context changes uncertainty.

Distribution before and after evidence

This gives us a useful pattern:

PRIOR DISTRIBUTION

↓

NEW EVIDENCE

↓

UPDATED DISTRIBUTION

The entire shape may change.

It may:

shift,
narrow,
widen,
develop new peaks.

This is central to Bayesian reasoning.

Discrete distributions

A discrete distribution applies when the random variable takes distinct values.

Examples include:

number of failures,
number of customers,
number of messages,
class labels.

For example:

X = number of machine failures today

might have probabilities over:

0, 1, 2, 3, ...

Each possible value can be assigned a probability.

Continuous distributions

A continuous distribution applies when the variable can take any value within a range.

Examples include:

temperature,
speed,
demand,
rainfall,
waiting time.

Instead of assigning probability to every exact value, we talk about probability over intervals.

For example:

What is the probability that tomorrow's temperature lies between 18°C and 20°C?

Density rather than point probability

For a continuous variable, the probability of one exact value is generally treated differently from a discrete case.

We instead describe a probability density.

The density tells us where probability is concentrated.

Probability is then associated with an interval under the curve.

Conceptually:

higher density → values in that region are more plausible

The normal distribution

One especially important distribution is the normal distribution.

It has the familiar bell shape:

highest probability near the centre,
decreasing probability further away,
symmetric tails.

It appears frequently because many small independent effects can combine to produce approximately normal variation.

Examples may include:

measurement noise,
biological variation,
some forecasting errors.

But it should not be used automatically.

Not everything is normal

Real-world data can have:

skew,
multiple peaks,
heavy tails,
sharp boundaries,
unusual shapes.

Income is rarely well described by a symmetric normal distribution.

Electricity prices may contain extreme spikes.

Waiting times may have long right tails.

Assuming a normal distribution when the real distribution is very different can hide important risk.

Heavy tails

A heavy-tailed distribution assigns more probability to extreme events than a normal distribution would.

This matters because extreme events occur more often than a simple bell-curve model might suggest.

Heavy tails appear in areas such as:

finance,
insurance,
internet traffic,
wealth,
system failures.

If we underestimate the tails, we may underestimate risk.

Rare events accumulate

Suppose an event has probability:

0.01%.

That seems tiny.

But imagine:

one billion opportunities

for it to occur.

We may still observe the event many times.

This is particularly important for large AI systems making enormous numbers of decisions.

Even extremely small tail probabilities can produce substantial numbers of failures at scale.

Bounded distributions

Some variables have natural limits.

For example:

battery state of charge

might lie between:

0% and 100%.

A probability distribution that assigns meaningful probability to:

-20%

or:

140%

would violate the physical constraints of the system.

The choice of distribution should respect what values are actually possible.

Probability over classes

Distributions also appear in classification.

Suppose an image model predicts:

cat: 0.70

dog: 0.20

fox: 0.10

This is a probability distribution over possible classes.

The probabilities sum to:

The model may output:

cat

as the final label.

But the full distribution tells us much more.

Confident versus uncertain classification

Compare:

Prediction A

cat: 0.98

dog: 0.01

fox: 0.01

with:

Prediction B

cat: 0.40

dog: 0.35

fox: 0.25

Both may select:

cat.

But the second prediction is much more uncertain.

The distribution captures that uncertainty.

Entropy

One way to think about how uncertain a distribution is involves entropy.

We do not need the full mathematics yet.

The intuition is:

probability concentrated on one outcome → low uncertainty,
probability spread fairly evenly across many outcomes → high uncertainty.

For example:

Distribution A

cat: 0.99

dog: 0.01

has low uncertainty.

Distribution B

cat: 0.50

dog: 0.50

has much higher uncertainty.

Entropy gives us a way to quantify that idea.

Language models output distributions

Large language models generate text by repeatedly constructing a probability distribution over possible next tokens.

Suppose the context is:

The capital of France is

The model may assign something conceptually like:

Paris: 0.97

Lyon: 0.01

Marseille: 0.005

other tokens: remaining probability

The distribution is highly concentrated.

Now consider:

My favourite colour is

The probability may be spread across many possible continuations.

The distribution is wider.

Sampling from a distribution

Once an AI model has a distribution over possible outputs, it still needs some way to select one.

One option is:

Always choose the most probable outcome.

Another is:

Sample from the distribution.

If:

A = 60%

B = 30%

C = 10%

then repeated sampling will produce A most often, but sometimes B or C.

This introduces variability into generation.

Deterministic versus probabilistic selection

Suppose a language model always selects the highest-probability token.

Its outputs may become:

predictable,
repetitive.

Sampling allows lower-probability alternatives to occasionally appear.

This can produce:

variety,
creativity,
different completions.

But it can also increase the chance of selecting poor or incorrect continuations.

The selection mechanism matters.

Temperature in generative AI

Many generative systems have a parameter often called temperature.

This adjusts how concentrated or spread out the token distribution becomes before selection.

Conceptually:

lower temperature

→ more probability concentrated on high-probability tokens

→ more predictable output

higher temperature

→ probability spread more widely

→ more varied output

This is not physical temperature.

It is a parameter controlling the shape of the output distribution.

Distributions can represent states

Suppose a robot is uncertain about its location.

It may maintain a distribution over possible positions.

Perhaps:

70% probability near Point A,
20% near Point B,
10% elsewhere.

The robot does not simply know:

one position.

It has a belief distribution over possible states.

This is central to robotics and state estimation.

A belief can be multimodal

Suppose a robot enters two visually identical corridors.

Its sensors cannot tell which corridor it is in.

The belief distribution may have:

one peak at Corridor A

and:

one peak at Corridor B.

The average location between the corridors may be somewhere the robot definitely is not.

This is another reason the full distribution matters.

Means can be misleading in multimodal systems

Suppose there is:

50% probability the robot is at x = 0

and:

50% probability it is at x = 100.

The expected position is:

x = 50.

But there may be essentially zero probability that the robot is at 50.

A single average has created a fictitious state.

In multimodal uncertainty, the full distribution is essential.

Forecast intervals

Instead of reporting only a point forecast, we can report an interval.

For example:

Tomorrow's demand is expected to be around 40 GW, with a 90% prediction interval from 37 to 44 GW.

The interval communicates a range of plausible outcomes.

Wider intervals generally indicate greater uncertainty.

Prediction intervals are not guarantees

90% prediction interval

does not mean:

The true value is guaranteed to be inside this range.

It means that if the forecasting method is well calibrated over repeated comparable cases, approximately 90% of future observations should fall inside its 90% intervals.

Some outcomes will still fall outside.

Quantiles

A distribution can also be summarised using quantiles.

For example:

50th percentile

is the median.

90th percentile

is a value that 90% of the distribution lies below.

Suppose the 95th percentile of demand is:

48 GW.

That means only around:

of the probability lies above 48 GW.

Quantiles are extremely useful for risk and planning.

Planning to a percentile

Suppose expected demand is:

40 GW.

But the 99th percentile is:

50 GW.

A system operator might decide to prepare for something closer to the upper percentile rather than the mean.

Why?

Because the cost of insufficient supply may be severe.

The appropriate percentile depends on:

reliability objectives,
cost,
consequences of failure.

Different users may care about different parts of the distribution

Suppose future electricity prices have a distribution.

A typical consumer may care mainly about:

expected cost.

A highly leveraged company may care about:

extreme high-price events.

A trader may care about:

volatility.

A battery operator may care about:

the probability of price spreads between periods.

The same distribution supports different decisions.

Distributions can move through time

Suppose we forecast demand 24 hours ahead.

At 09:00, the distribution is broad.

At 17:00, we have more recent data and improved weather forecasts.

The distribution may narrow.

As the event approaches, uncertainty often decreases.

But not always.

Unexpected events can widen it again.

Forecast horizons change distributions

Consider demand forecasts:

5 minutes ahead

1 hour ahead

1 day ahead

1 year ahead.

As the horizon increases, more things can change.

The distribution often becomes wider.

So prediction should always specify:

A distribution for what time horizon?

Spatial distributions

Uncertainty can also exist over location.

Suppose a weather system predicts where a storm may make landfall.

Instead of one exact point, it may produce a distribution over possible locations.

Some areas have:

high probability,
moderate probability,
low probability.

The same principle applies to:

wildfire spread,
vehicle position,
disease outbreaks,
renewable generation.

Spatiotemporal distributions

Many real forecasts involve uncertainty across both:

space

and:

time.

For example:

Where will the storm be at 18:00 tomorrow?

The output is not necessarily:

one location.

It may be a probability field over many possible locations.

So:

P(state at location x, time t)

can describe uncertainty about a system evolving through space and time.

Joint distributions

Suppose:

D = electricity demand

and:

W = wind generation.

We care about both simultaneously.

A joint distribution describes probabilities for combinations such as:

high D + high W

high D + low W

low D + high W

and so on.

This matters because system risk may depend on combinations of uncertain variables.

Marginal distributions

From a joint distribution, we can sometimes focus on one variable alone.

For example:

P(D)

without considering the exact value of W.

This is called a marginal distribution.

The idea is useful because complex systems may contain many variables, while a particular question may focus on only one.

Conditional distributions

We can also ask:

P(D | W = low)

meaning:

What is the distribution of demand given that wind generation is low?

Conditional distributions allow information about one variable to reshape our belief about another.

This will become central in Bayesian reasoning.

Distributions can be learned from data

Suppose we have 10 years of temperature measurements.

We can examine:

how often different temperatures occurred,
how spread out values were,
whether the distribution was symmetric,
whether it changed by season.

This gives us an empirical distribution.

Models can then use historical data to estimate distributions for future outcomes.

But historical distributions can shift

Suppose the historical demand distribution was:

35–45 GW.

Then millions of electric vehicles enter the system.

Future demand may have a different:

centre,
spread,
shape.

A distribution learned from history is not guaranteed to remain valid forever.

Again:

The future may not be distributed like the past.

Model uncertainty and outcome uncertainty

There is another subtle distinction.

Suppose tomorrow's demand is uncertain because people behave unpredictably.

That is uncertainty about the outcome.

But perhaps we are also unsure which forecasting model is correct.

That is uncertainty about the model.

The final predictive distribution may need to reflect both.

Mixtures of distributions

Suppose two plausible models exist.

Model A predicts:

demand centred around 38 GW.

Model B predicts:

demand centred around 44 GW.

If we are uncertain which model is correct, the overall distribution might combine both possibilities.

This can produce a multimodal distribution.

Uncertainty about models can therefore become uncertainty about outcomes.

Distributions and calibration

A predicted distribution should correspond to reality over repeated observations.

If a model says:

90% of outcomes should fall inside this interval,

then roughly 90% should.

If only 50% do, the model is overconfident.

If 100% do and the intervals are extremely wide, the model may be underconfident or uninformative.

A useful forecast should be:

calibrated

and:

sharp enough to be useful.

Sharpness

A model could guarantee excellent coverage by always predicting:

Tomorrow's temperature will be between -100°C and +100°C.

That interval would probably contain the answer.

But it is not useful.

We therefore want distributions that are:

well calibrated,
as concentrated as the evidence justifies.

This balance is important in probabilistic forecasting.

A distribution can be wrong even if the average is right

Suppose actual demand is highly volatile.

A model correctly predicts:

average demand = 40 GW.

But it assumes almost no variation.

The mean is correct.

The distribution is wrong.

If we use the model to decide reserve requirements, this could be disastrous.

Evaluation must therefore consider more than point accuracy.

Distributions allow better scoring

When a model predicts a complete distribution, we can score not just whether one point was close.

We can ask:

Did the model assign high probability to what actually happened?
Was it too confident?
Was it too vague?
Did the prediction intervals have correct coverage?

This leads to probabilistic scoring rules, which we will examine later.

Distribution matters for optimisation

Suppose a hospital wants to schedule beds.

Expected admissions tomorrow are:

50.

But the distribution shows:

10% chance of fewer than 40,
60% chance between 40 and 55,
30% chance above 55.

The decision should not necessarily schedule exactly:

50 beds.

It depends on:

cost of spare capacity,
cost of turning patients away,
desired reliability.

The distribution feeds into the optimisation problem.

Prediction plus consequences

This gives us the emerging structure:

PROBABILITY DISTRIBUTION

↓

POSSIBLE OUTCOMES

↓

CONSEQUENCES OF EACH OUTCOME

↓

DECISION

Prediction is about describing the distribution.

Decision-making is about choosing what to do given that distribution.

The distinction will become increasingly important later.

Distributions and shared resources

Suppose a data centre has:

100 units of compute capacity.

Tomorrow's demand is uncertain.

A distribution might tell us:

20% chance demand is below 70,
50% chance demand is 70–100,
30% chance demand exceeds capacity.

Now we have a service-design problem.

Do we:

add capacity,
ration,
queue,
prioritise users,
accept lower reliability?

Uncertainty and scarcity interact.

Distributions describe possible futures

We can now refine the course's central timeline.

Instead of:

PRESENT → PREDICTED FUTURE

we have:

PRESENT STATE

↓

MODEL

↓

DISTRIBUTION OVER POSSIBLE FUTURES

↓

DECISION

↓

ACTION

↓

ONE FUTURE IS REALISED

The distribution exists before the outcome.

Reality eventually selects one path.

The realised future becomes new data

Suppose we predicted:

P(D = 40 GW) = high

but actual demand becomes:

46 GW.

That observation now becomes part of the dataset.

We can compare:

predicted distribution

with:

realised outcome.

Then update the model.

So:

DISTRIBUTION

↓

REALISATION

↓

ERROR / EVIDENCE

↓

MODEL UPDATE

The feedback loop continues.

Ask about the whole distribution

Whenever you encounter a prediction, ask:

What is the random variable?
What values are possible?
What is the centre of the distribution?
How wide is it?
Is it symmetric?
Is it skewed?
Does it have one peak or several?
How heavy are the tails?
What are the relevant percentiles?
Are there hard physical limits?
Does the distribution change with context?
Does it change through time?
Is the model calibrated?
Are rare outcomes important for the decision?

These questions provide a much richer understanding than asking only:

What is the prediction?

The central idea

A probability distribution describes how uncertainty is spread across the possible values of a random variable.

It tells us:

what could happen

and:

how plausible each possibility is.

The centre matters.

The spread matters.

The tails matter.

The shape matters.

And sometimes the most important part of the distribution is not the most likely outcome at all.

A forecast is not one future. It is a structured description of multiple possible futures.

This gives us a richer progression:

RANDOM VARIABLE

↓

POSSIBLE VALUES

↓

PROBABILITY DISTRIBUTION

↓

CENTRE + SPREAD + TAILS + SHAPE

↓

DECISION UNDER UNCERTAINTY

In the next lesson, we will look at one of the most common ways of summarising a distribution:

expected value — the probability-weighted average of possible outcomes, why it is useful, and why it can also be dangerously incomplete.