Probability as a language for uncertainty

In the previous lesson, we established a simple idea:

Uncertainty is unavoidable.

We do not observe the world perfectly.

We do not know the future.

Our models are incomplete.

Our measurements contain error.

And even when we have enormous amounts of data, several different futures may remain possible.

If uncertainty is unavoidable, we need some way to describe it.

That is what probability gives us.

Probability is a mathematical language for describing uncertainty.

It allows us to move beyond vague statements such as:

maybe,
probably,
unlikely,
almost certainly.

Instead, we can attempt to express how strongly the available evidence supports different possibilities.

From possible to probable

Suppose tomorrow could be:

sunny,
cloudy,
rainy.

Simply listing these possibilities tells us something.

But it does not tell us whether they are equally plausible.

Perhaps our forecast is:

Sunny: 10%

Cloudy: 30%

Rainy: 60%

Now we have represented more than the possible futures.

We have represented our belief about their relative likelihood.

Conceptually:

POSSIBLE FUTURES

↓

PROBABILITIES

↓

RELATIVE PLAUSIBILITY OF EACH FUTURE

Probability gives structure to uncertainty.

Probability ranges from 0 to 1

Probabilities are usually represented using numbers between:

0 and 1

where:

0 = impossible

and:

1 = certain.

So:

P(rain) = 0.7

means:

We assign a probability of 70% to rain.

Percentages are simply another way of writing probabilities:

0.7 = 70%

0.25 = 25%

0.01 = 1%

The mathematics is simple.

The interpretation requires more care.

What does 70% probability actually mean?

Suppose a weather forecast says:

There is a 70% probability of rain tomorrow.

What does that mean?

It does not mean:

It will rain for 70% of the day.

It does not necessarily mean:

70% of the region will receive rain.

And it certainly does not mean:

The forecaster is 70% sure in some vague emotional sense.

A useful interpretation is:

Among situations for which we assign a probability of 70%, the event should occur approximately 70% of the time.

If a forecasting system repeatedly makes well-calibrated:

70% predictions

then, over many comparable cases, roughly:

7 out of 10

should occur.

This connects probability to calibration.

A single event still happens or does not happen

Suppose the probability of rain tomorrow is:

70%.

Tomorrow arrives.

It either:

rains

or:

does not rain.

We do not observe 70% rain.

The probability described our uncertainty before the outcome became known.

This is an important distinction:

Probability describes uncertainty about outcomes, not partial occurrence of an outcome.

Probability before and after observation

Before rolling a fair die:

P(rolling a 6) = 1/6

After rolling the die and observing a 6:

the uncertainty about that particular roll has disappeared.

The event happened.

Probability is therefore connected to what information is available at a particular moment.

As information changes, probabilities can change.

Probability depends on information

Suppose we ask:

What is the probability that a person is carrying an umbrella?

Without any additional information, perhaps our estimate is:

20%.

Now we learn:

It is raining heavily.

Our estimate might become:

80%.

The person has not changed between the two calculations.

Our information changed.

So our probability changed.

This is a central idea:

Probability is often conditional on what we know.

Probability is not always a physical property

Some probabilities appear naturally connected to repeated physical processes.

For example:

rolling dice,
radioactive decay,
manufacturing defects.

But many probabilities describe our uncertainty about things that happen only once.

For example:

What is the probability that a particular company will fail next year?

What is the probability that this patient has a disease?

What is the probability that this election result occurs?

What is the probability that this image contains a pedestrian?

In these cases, probability can represent our degree of uncertainty given the available evidence.

Two interpretations of probability

There are several philosophical interpretations of probability.

Two are particularly useful for this course.

Frequentist interpretation

Probability is associated with what happens over many repeated trials.

If a fair coin is flipped many times, the proportion of heads should approach:

50%.

Probability describes the long-run frequency of an event.

Bayesian interpretation

Probability can also represent a degree of belief given available information.

We might say:

Given everything we currently know, there is an 80% probability that this patient has the disease.

The patient either has the disease or does not.

The probability represents our uncertainty about which state is true.

Both perspectives are useful.

Probability does not mean guessing

Suppose a model says:

There is a 62% probability of failure.

That should not mean:

I picked 62% because it felt plausible.

Ideally, the probability comes from:

observations,
statistical models,
assumptions,
previous evidence.

Probability provides a formal language.

But the quality of the probability still depends on the quality of the reasoning and evidence behind it.

Experiments and outcomes

Probability often begins by defining an experiment.

An experiment is simply a process whose outcome is uncertain.

Examples include:

flipping a coin,
rolling a die,
observing tomorrow's weather,
measuring whether a machine fails,
predicting whether a customer defaults.

The individual result is an outcome.

Sample spaces

The collection of all possible outcomes is called the sample space.

For a coin:

S = {Heads, Tails}

For a six-sided die:

S = {1, 2, 3, 4, 5, 6}

For tomorrow's simplified weather:

S = {Sunny, Cloudy, Rainy}

The sample space defines the possible worlds we are considering.

This already involves modelling.

We decide which outcomes matter.

The sample space is a representation

Suppose tomorrow's weather is represented as:

{Sunny, Cloudy, Rainy}

Reality is obviously more complicated.

Tomorrow might involve:

rain in the morning,
sunshine in the afternoon,
different conditions in different locations,
different rainfall intensities.

We simplified reality into three categories.

So even before assigning probabilities, we have already created a model.

Probability operates over the possibilities we choose to represent.

If an important possibility is absent from the model, probability cannot magically restore it.

Events

An event is one or more outcomes we care about.

Suppose we roll a die.

The sample space is:

{1, 2, 3, 4, 5, 6}

The event:

Roll an even number

contains:

{2, 4, 6}

The event:

Roll more than 4

contains:

{5, 6}

Probability allows us to assign likelihoods to these events.

Equally likely outcomes

For a fair six-sided die, each outcome has probability:

1/6.

The probability of rolling an even number is therefore:

3/6 = 1/2.

But this calculation relies on an assumption:

The die is fair.

If the die is weighted, the outcomes are no longer equally likely.

This illustrates an important principle:

Probabilities depend on assumptions about the process generating the data.

Probabilities must add up

If our list of outcomes is complete and mutually exclusive, their probabilities must sum to:

Suppose:

P(Sunny) = 0.2

P(Cloudy) = 0.5

P(Rainy) = 0.3

Then:

0.2 + 0.5 + 0.3 = 1

Why?

Because one of the represented outcomes must occur.

The total probability represents all possibilities within the model.

The complement

Suppose:

P(rain) = 0.3

Then:

P(no rain) = 0.7

because:

P(rain) + P(no rain) = 1

The event:

not A

is called the complement of A.

So:

P(not A) = 1 - P(A)

This simple rule is surprisingly useful.

Multiple events

Suppose we roll a die.

Let:

A = roll an even number

and:

B = roll a number greater than 3.

Then:

A = {2, 4, 6}

and:

B = {4, 5, 6}.

Some outcomes belong to both events:

{4, 6}.

This overlap is called the intersection.

It means:

A and B both occur.

"And" versus "or"

Probability requires us to be precise with language.

A AND B

means both events occur.

A OR B

means at least one occurs.

These sound simple, but confusing them can produce very different probabilities.

Later we will use these ideas when discussing:

conditional probability,
Bayesian reasoning,
combinations of risks.

Independent events

Suppose we flip a fair coin twice.

The result of the first flip does not affect the probability of the second.

These events are independent.

So:

P(Heads on second flip | Heads on first flip) = 0.5

The first result gives us no useful information about the second.

Independence is an assumption

Real-world events are often not independent.

Suppose a machine fails today.

Perhaps that makes failure tomorrow more likely because the underlying equipment is deteriorating.

Or suppose electricity demand is high at 17:00.

That may make high demand at 17:30 more likely.

Time-series observations are often related.

Assuming independence when events are actually connected can produce very poor models.

Conditional probability

Suppose:

1% of the population has a particular disease.

That gives us:

P(disease).

Now suppose a person receives a positive test result.

We want something different:

P(disease | positive test)

Read this as:

The probability of disease given that the test is positive.

The symbol:

means:

given that.

Conditional probability is one of the most important ideas in probability.

New information changes probability

Before the test, perhaps:

P(disease) = 1%.

After a positive test, perhaps:

P(disease | positive) = 40%.

The underlying health state of the person did not suddenly change because we performed the calculation.

Our information changed.

So our belief changed.

This is the foundation of Bayesian reasoning.

Prior beliefs and evidence

We can think of reasoning as:

WHAT WE BELIEVED BEFORE

↓

NEW EVIDENCE

↓

WHAT WE BELIEVE NOW

In Bayesian terminology:

PRIOR

↓

EVIDENCE

↓

POSTERIOR

This is a powerful framework because intelligent systems constantly receive new information.

Base rates matter

Suppose a disease is extremely rare.

A test might be highly accurate.

Yet a positive test may still not mean the disease is almost certain.

Why?

Because we need to consider how common the disease was before seeing the test.

This is the base rate.

Ignoring base rates is one of the most common errors humans make when reasoning about probability.

A simple medical example

Suppose:

1 in 1,000 people

has a disease.

Imagine a test that correctly identifies most people with the disease but also produces a small number of false positives.

If we test:

100,000 people

perhaps around:

100

actually have the disease.

Even a very small false-positive rate applied to the remaining:

99,900 people

can produce many false positives.

So:

A positive result is not interpreted in isolation.

We combine:

how common the condition was beforehand,
how reliable the test is,
the evidence we observed.

This is exactly the kind of reasoning probability allows.

Probability helps us update beliefs

Suppose a robot believes there is:

20% probability

that an obstacle is behind a wall.

Its radar detects something.

The probability rises to:

70%.

Its camera then sees part of an object.

The probability rises to:

95%.

Conceptually:

PRIOR BELIEF

↓

OBSERVATION

↓

UPDATED BELIEF

↓

NEW OBSERVATION

↓

UPDATED BELIEF AGAIN

This is how uncertain knowledge can evolve over time.

Probability and state estimation

Recall the idea that:

observation ≠ state.

A robot does not directly observe its complete state.

It receives:

camera images,
GPS signals,
radar,
wheel measurements.

From these observations, it estimates:

position,
velocity,
nearby objects.

Because measurements are imperfect, the state estimate is uncertain.

So instead of saying:

the robot is exactly here

we might say:

The robot is most likely here, with some probability assigned to nearby positions.

Probability becomes part of the machine's internal representation of the world.

Probability distributions

Suppose we predict tomorrow's electricity demand.

Instead of:

Demand = 40 GW

we might represent:

38 GW: relatively unlikely,
39 GW: plausible,
40 GW: highly plausible,
41 GW: plausible,
42 GW: relatively unlikely.

Together, these probabilities form a probability distribution.

A distribution describes how probability is spread across possible values.

Distributions are richer than single predictions

Compare:

Forecast A

Demand tomorrow = 40 GW.

with:

Forecast B

Demand is centred around 40 GW, with most plausible outcomes between 38 and 42 GW.

Forecast B contains information about uncertainty.

This matters because decisions may depend not only on:

what is most likely

but also:

what else could reasonably happen.

Probability can describe continuous quantities

Not everything has a small number of discrete outcomes.

Temperature might be:

19.1°C

19.2°C

19.21°C

and so on.

Position, speed, demand and rainfall can vary continuously.

For continuous quantities, we often describe uncertainty using a probability density.

The underlying mathematics differs slightly from the simple die example.

But the conceptual idea is the same:

Some ranges of values are considered more plausible than others.

Probability does not guarantee an outcome

Suppose a model predicts:

90% probability of success.

The project fails.

Was the model necessarily wrong?

No.

A 90% event should still fail approximately:

10% of the time.

Likewise, if a model predicts:

10% probability of an event

and the event occurs, that does not automatically mean the prediction was bad.

Unlikely events happen.

We cannot judge probabilistic forecasts from one event

Suppose a weather forecaster says:

10% chance of rain.

It rains.

Someone says:

The forecast was wrong.

But that conclusion is too simple.

To evaluate a probabilistic forecaster, we need many predictions.

Among all days assigned approximately:

10% probability of rain

did it rain roughly:

10% of the time?

This is why probabilistic predictions need different evaluation methods from deterministic predictions.

Calibration

A probabilistic model is well calibrated if its probabilities correspond to observed frequencies over many comparable cases.

If a model makes:

1,000 predictions at 80% confidence

then approximately:

800

should be correct.

Not necessarily exactly 800.

Random variation still exists.

But over enough predictions, the relationship should be close.

Confidence is something we can test

This is important for AI.

A system should not merely output:

I am 95% confident.

We can examine whether predictions made with 95% confidence are actually correct approximately 95% of the time.

This turns confidence from a vague statement into something measurable.

Probability is not certainty with decimal places

A model outputting:

0.8734

can look extremely precise.

But numerical precision is not the same as knowledge.

The true uncertainty may be much larger.

Perhaps:

the training data is limited,
the environment has changed,
important variables are missing,
the model itself is misspecified.

Writing more decimal places does not make uncertainty disappear.

Probabilities come from models

Suppose an AI estimates:

P(default) = 0.12.

That number is not floating independently in nature.

It comes from some combination of:

data,
assumptions,
model structure,
parameters,
features,
training procedures.

Change those inputs and the probability may change.

So probabilities should always be understood in context.

Different models can assign different probabilities

Suppose two weather models predict:

Model A: 30% chance of rain

Model B: 60% chance of rain.

They are considering the same future.

Why do they disagree?

Perhaps they use:

different data,
different assumptions,
different model structures,
different estimates of the current state.

Probability does not eliminate disagreement.

It gives us a language for expressing it.

Humans use probability intuitively

We constantly make probabilistic judgments.

We say:

The train will probably be late.

I doubt that shop is open.

That road is usually busy.

This candidate is likely to win.

We rarely calculate exact probabilities.

But we are implicitly reasoning about uncertainty.

Formal probability gives us tools for making that reasoning more consistent.

Humans are not naturally good at probability

Our intuition can be misleading.

We may:

overweight dramatic events,
ignore base rates,
see patterns in randomness,
become overconfident,
confuse possibility with probability,
treat recent events as unusually likely.

This is one reason formal probability matters.

It disciplines intuition.

Rare does not mean impossible

Suppose an event has probability:

0.001.

That means:

0.1%.

It is unlikely.

But if there are:

1,000,000 opportunities

for the event to occur, we may expect it to happen many times.

This matters enormously in large systems.

Scale changes the meaning of rare events

Suppose a data centre contains millions of components.

A component failure with a tiny probability may still occur regularly somewhere in the system.

Suppose an AI service makes billions of predictions.

Even an extremely low error rate can generate many errors.

So:

Small probability × enormous number of opportunities can still produce many events.

This is why scale matters when evaluating AI systems.

Probability and risk

Suppose two events each have probability:

1%.

Event A causes:

a minor inconvenience.

Event B causes:

catastrophic system failure.

Their probabilities are identical.

Their risks are not.

Probability describes likelihood.

Decision-making must also consider consequence.

Conceptually:

RISK ≈ PROBABILITY × CONSEQUENCE

This is only a simplified representation, but it captures an important idea.

Probability does not tell us what to do

Suppose an AI predicts:

There is a 70% probability that Patient A benefits from treatment.

That is a prediction.

It does not automatically tell us:

Give Patient A the treatment.

We still need to consider:

side effects,
alternative treatments,
resource availability,
other patients,
costs,
fairness,
uncertainty.

Probability informs decisions.

It does not replace them.

The same probability can produce different decisions

Suppose there is a:

20% probability of rain.

For a walk in the park, you may ignore it.

For an outdoor wedding, you may arrange a tent.

For a rocket launch, a 20% probability of dangerous weather may cancel the mission.

The probability is the same.

The decision changes because:

objectives and consequences differ.

Probability separates prediction from preference

This distinction is extremely important.

Probability asks:

What is likely to happen?

Preference or utility asks:

Which outcomes do we prefer?

Decision-making asks:

Given what is likely and what we care about, what should we do?

These are different questions.

Later in the course, this distinction will become:

PREDICTION

↓

PROBABILITIES OF POSSIBLE FUTURES

↓

OBJECTIVES + CONSTRAINTS

↓

DECISION

Probability and machine learning

Many machine-learning models ultimately produce probabilities.

A classifier might output:

cat: 0.82

dog: 0.15

fox: 0.03

These values describe how the model distributes belief across possible labels.

The final application may simply display:

CAT

because that is the most likely class.

But underneath that decision is a probability distribution.

The most likely answer can hide uncertainty

Compare:

Prediction A

Cat: 99%

Dog: 1%

with:

Prediction B

Cat: 51%

Dog: 49%

Both systems may output:

CAT.

But these predictions are not remotely equivalent.

The first is highly confident.

The second is almost a coin toss.

Reducing both to the same label destroys important information.

Language models use probability too

Large language models provide a striking example.

Given some context, an LLM estimates probabilities over possible next tokens.

Conceptually:

CONTEXT

↓

POSSIBLE NEXT TOKENS

↓

PROBABILITY DISTRIBUTION

For example:

The capital of France is ...

might produce something conceptually like:

Paris: very high probability

Lyon: very low probability

London: extremely low probability

The model then selects a token and repeats the process.

Generation is repeated probabilistic prediction

After selecting one token, the context changes.

Then the model predicts again:

CONTEXT

↓

TOKEN PROBABILITIES

↓

SELECT TOKEN

↓

UPDATED CONTEXT

↓

NEW TOKEN PROBABILITIES

↓

SELECT TOKEN

↓

...

An apparently coherent paragraph can emerge from repeated predictions over possible next tokens.

This is one reason probability is foundational to understanding modern AI.

Probability enables alternatives

A generative model does not necessarily have only one possible continuation.

Suppose the prompt is:

Once upon a...

Possible next tokens might include:

time,
day,
night,
world.

Each receives some probability.

Different choices can lead to completely different future sequences.

So generation can be understood as movement through a branching space of possible futures.

The future as a probability tree

Imagine the present moment.

Several things might happen next.

Each of those futures creates another set of possibilities.

Conceptually:

PRESENT

↓

FUTURE A

→ A1

→ A2

FUTURE B

→ B1

→ B2

FUTURE C

→ C1

→ C2

Probability gives us a way to assign relative likelihoods to the branches.

As time progresses, one path becomes reality.

The others become unrealised possibilities.

Probability connects the past to possible futures

We can now extend the central course framework.

PAST

↓

DATA

↓

INFERENCE

↓

ESTIMATED PRESENT STATE

↓

PROBABILITY DISTRIBUTION OVER POSSIBLE FUTURES

↓

DECISION

↓

ACTION

↓

ONE FUTURE BECOMES REALITY

↓

NEW DATA

The future is not represented as a single predetermined point.

It is represented as a set of possibilities with different probabilities.

Probability changes as the world changes

Suppose at 08:00:

P(rain at noon) = 30%.

At 10:00, new satellite observations arrive.

Now:

P(rain at noon) = 70%.

At 11:45, dark clouds are overhead.

Now:

P(rain at noon) = 95%.

The future event is getting closer.

New information arrives.

Our probability changes.

This is exactly what an intelligent system should do:

observe → update → predict again.

Probability is stateful

Our beliefs at one moment depend on:

what we previously believed,
what we have observed since.

So probabilistic reasoning naturally fits into systems that evolve through time.

Conceptually:

BELIEF AT TIME t

NEW OBSERVATION

↓

BELIEF AT TIME t + 1

This idea will become central when we study:

Bayesian reasoning,
state estimation,
sequential decision-making,
robotics.

Uncertainty can be represented rather than hidden

The most important philosophical point is simple.

We do not need to pretend uncertainty does not exist.

Instead of:

We don't know, therefore we cannot reason.

probability allows us to say:

We do not know exactly, but some possibilities are better supported than others.

That is an enormous conceptual step.

Probability is not truth

A probability is not the event itself.

It is a representation of uncertainty about the event.

A forecast of:

70% rain

is not rain.

A model output of:

90% probability of disease

is not a diagnosis by itself.

A prediction of:

80% probability of default

is not a default.

The distinction between:

model

and:

reality

still applies.

Probability can be wrong

A model may assign:

99% probability

to something that does not happen.

That alone does not prove the model is bad.

But if events assigned 99% probability repeatedly fail to occur, something is wrong.

Perhaps:

the model is poorly calibrated,
the data is biased,
the environment changed,
assumptions are incorrect.

Probability estimates themselves must be evaluated.

The map still is not the territory

Probability gives us a richer map.

Instead of saying:

THE FUTURE IS X

we say:

THESE FUTURES ARE POSSIBLE, WITH THESE ESTIMATED PROBABILITIES.

That is better.

But it is still a model.

There may be possibilities we did not include.

There may be uncertainty we cannot quantify.

There may be unknown unknowns.

So probability helps us reason about uncertainty.

It does not make us omniscient.

The central idea

Probability gives us a language for moving from:

I don't know

to:

Here are the possibilities, and here is how plausible each currently appears.

It allows us to represent:

uncertainty,
evidence,
possible outcomes,
changing beliefs,
risk,
confidence.

And it allows intelligent systems to reason without pretending the world is certain.

The core progression is:

OBSERVATION

↓

UNCERTAINTY

↓

POSSIBLE STATES

↓

PROBABILITIES

↓

NEW EVIDENCE

↓

UPDATED PROBABILITIES

Probability therefore provides the mathematical bridge between:

what we have observed

and:

what we believe about what we cannot directly know.

In the next lesson, we will give uncertain quantities themselves a mathematical representation.

These are called random variables.