Module 3 — Uncertainty, probability and inference
Lesson 7 of 13
Conditional probability
Probability changes when information changes.
Suppose we ask:
What is the probability that someone is carrying an umbrella?
Without any other information, perhaps the probability is relatively low.
Now suppose we learn:
It is raining heavily.
Our estimate changes.
The person has not changed.
The weather may not even have changed between the two questions.
What changed was what we knew.
This is the idea behind conditional probability.
Conditional probability describes the probability of an event given that we know something else is true.
It allows us to represent how evidence changes uncertainty.
"Given that"
Conditional probability is usually written:
P(A | B)
and read as:
The probability of A given B.
The vertical line:
|
means:
given that.
For example:
P(Umbrella | Rain)
means:
What is the probability that someone has an umbrella given that it is raining?
This is different from:
P(Umbrella)
which asks about umbrella use without specifying the weather.
Information changes the relevant population
Suppose we observe 1,000 people.
Of those:
- 200 are outside while it is raining,
- 160 of those 200 carry umbrellas.
Then:
P(Umbrella | Rain) = 160 / 200 = 0.8
or:
80%.
Notice that we did not divide by all 1,000 people.
Once we know it is raining, we restrict our attention to the subset of cases where:
Rain = true.
Conditional probability asks:
Within the world where B is true, how often is A also true?
The conditional probability formula
For two events A and B:
P(A | B) = P(A and B) / P(B)
provided:
P(B) > 0.
In words:
Take the probability that both A and B happen, and divide by the probability that B happens.
The key idea is not the algebra.
It is the change in reference population.
We are no longer considering all possible cases.
We are considering only cases where B is true.
A simple example
Suppose a class contains:
100 students.
Of those:
- 60 study mathematics,
- 30 study physics,
- 20 study both.
What is:
P(Physics | Mathematics)?
We restrict ourselves to the:
60 mathematics students.
Among them:
20 also study physics.
So:
P(Physics | Mathematics) = 20 / 60 = 1/3
The question is not:
What fraction of all students study physics?
It is:
Among mathematics students, what fraction also study physics?
Direction matters
Conditional probability is directional.
In general:
P(A | B) ≠ P(B | A)
This is extremely important.
Suppose:
Most people with a rare disease test positive.
That does not automatically mean:
Most people who test positive have the disease.
Those statements sound similar.
Mathematically, they are different probabilities.
A medical example
Let:
D = patient has disease
and:
+ = test is positive.
Then:
P(+ | D)
means:
Probability the test is positive given that the patient has the disease.
This measures something about the test.
But what the patient usually wants to know is:
P(D | +)
meaning:
Probability the patient has the disease given that the test was positive.
These are not the same thing.
Confusing them is one of the most common errors in probabilistic reasoning.
Why the two probabilities differ
Suppose a disease is extremely rare.
Imagine:
1 in 1,000 people
has it.
A test may be very good at detecting the disease.
But even a small false-positive rate applied to the other:
999 people
can produce many positive results among people who do not have the disease.
So:
P(+ | D)
can be very high
while:
P(D | +)
is much lower.
The background prevalence matters.
Base rates matter
The probability of disease before receiving the test result is called the base rate or prior probability.
Suppose:
P(D) = 0.001
Then a positive test provides evidence.
But that evidence must be combined with the fact that the disease was initially very rare.
This is why conditional probability is foundational to Bayesian reasoning.
Conditional probability updates the world we consider
Imagine the full population as one large set.
Then learn:
B is true.
Everything outside B becomes irrelevant for this particular conditional question.
Within B, we ask how much also belongs to A.
Conceptually:
ALL POSSIBLE CASES
↓
OBSERVE B
↓
KEEP ONLY CASES CONSISTENT WITH B
↓
ASK HOW OFTEN A OCCURS
That is conditional probability.
Evidence narrows possibilities
Recall our course framework:
POSSIBLE STATES
↓
NEW OBSERVATION
↓
FEWER PLAUSIBLE STATES
Suppose a robot could be in:
- Room A,
- Room B,
- Room C.
Before observing anything:
P(A) = 0.33
P(B) = 0.33
P(C) = 0.34
Now its camera sees:
a red door.
Suppose only Rooms A and C have red doors.
Room B becomes much less plausible.
The observation changes the probability distribution over states.
Conditional distributions
Conditional probability applies not only to single events.
We can also talk about a complete conditional distribution.
Suppose:
D = electricity demand
and:
T = outside temperature.
We might have one distribution:
P(D)
describing demand without any specific temperature information.
Then we learn:
T = -2°C.
Now we consider:
P(D | T = -2°C).
The entire demand distribution may shift upward.
Context changes predictions
Suppose a model predicts journey time.
Without additional information:
expected journey = 35 minutes.
Now we learn:
It is Monday at 08:30.
The distribution changes.
Then:
It is raining.
It changes again.
Then:
There is an accident on the route.
It changes again.
Prediction is therefore often conditional on context.
Machine learning is full of conditional probabilities
Suppose a classifier predicts whether an email is spam.
The model is effectively trying to estimate something like:
P(Spam | Email features).
Given:
- words,
- sender,
- formatting,
- links,
what is the probability the email is spam?
Likewise:
P(Disease | Symptoms)
P(Default | Financial history)
P(Pedestrian | Image)
Conditional probability sits underneath much of machine learning.
Regression is conditional too
Suppose we predict house price.
We might ask:
What is the expected house price given its features?
Conceptually:
E[Price | Size, Location, Age, ...]
The prediction is conditioned on the information supplied to the model.
The richer the relevant information, the more specific the conditional prediction can become.
Prediction without context is usually weak
Suppose we ask:
What will electricity demand be?
That question is underspecified.
Demand depends on:
- location,
- time,
- temperature,
- day,
- previous demand,
- many other variables.
A meaningful forecast is more like:
What is the distribution of demand at this location and time, given everything we currently know?
Prediction is fundamentally conditional.
Time creates conditioning
Suppose:
X(t)
is electricity demand at time t.
We may want:
P(X(t+1) | X(t))
meaning:
What is the probability distribution of the next demand value given the current value?
Or perhaps:
P(X(t+1) | X(t), X(t-1), Temperature).
Time-series models condition future predictions on historical observations.
The past becomes information
This connects directly to the philosophical foundation of the course.
We cannot directly observe the future.
We have:
past observations
and:
present conditions.
So forecasting becomes:
P(Future | Past, Present).
This may be one of the simplest ways to describe prediction mathematically.
Prediction is conditional reasoning through time.
The present state can summarise the past
In some systems, a sufficiently informative state representation may contain everything from the past that matters for predicting the future.
Then:
P(Future | Entire past)
can sometimes be simplified to something like:
P(Future | Current state).
This is an important idea in state-space models.
The state acts as a compressed representation of relevant history.
The Markov idea
A system has the Markov property when the future depends on the present state rather than directly on the entire history, once the present state is known.
Conceptually:
PAST → PRESENT STATE → FUTURE
If the state is complete enough:
Knowing the present state makes the distant past unnecessary for predicting the next state.
This is a powerful modelling idea.
The quality of the state representation matters
Suppose we model a moving vehicle using only:
position.
Can we predict where it will be in one second?
Not very well.
We also need:
velocity.
Perhaps:
acceleration.
So a better state might be:
x = [position, velocity, acceleration]
Once the relevant state is represented, future prediction becomes easier.
Conditional probability therefore connects naturally to state-vector machines.
Hidden states
Often we cannot directly observe the state.
Suppose a machine has an internal health condition:
healthy
degrading
near failure
We cannot observe it directly.
We observe:
- vibration,
- temperature,
- noise.
We want:
P(State | Observations).
This is another conditional probability problem.
Observations provide evidence about hidden state
Suppose:
P(machine failing soon) = 5%
before observing anything unusual.
Then vibration increases sharply.
Perhaps:
P(machine failing soon | high vibration) = 40%.
Then temperature also rises.
Perhaps the probability increases further.
The machine's hidden physical state did not necessarily change at each calculation.
Our belief about it changed as evidence accumulated.
Conditional probability and sensor fusion
Suppose a robot wants to know whether an object ahead is a pedestrian.
Camera evidence suggests:
70%.
Radar suggests an object of pedestrian-like size.
Lidar provides additional shape information.
The system can combine these pieces of evidence.
Conceptually:
P(Pedestrian | Camera, Radar, Lidar).
Multiple observations reduce uncertainty about hidden state.
But evidence may not be independent
Suppose two sensors both depend on the same camera image.
Treating them as independent pieces of evidence would double-count information.
This is an important issue.
Conditional relationships determine how information should be combined.
Independence
Two events A and B are independent if knowing B does not change the probability of A.
In that case:
P(A | B) = P(A).
For example, two independent fair coin flips.
Knowing:
The first flip was Heads
does not change:
P(second flip is Heads) = 0.5.
The first observation gives no information about the second.
Dependence
Now suppose:
A = high electricity demand at 18:00
and:
B = high electricity demand at 17:30.
These are likely dependent.
If demand is already high at 17:30, high demand at 18:00 may become more probable.
So:
P(A | B) > P(A)
may hold.
Temporal dependence is one reason past data helps predict future data.
Correlation and conditional probability
Suppose two variables appear correlated.
Conditional probability can help investigate whether the relationship persists once we account for other variables.
For example:
P(Disease | Exercise)
might differ from:
P(Disease | Exercise, Age).
Age may explain part of the original relationship.
Conditioning helps us examine relationships more carefully.
Conditioning can reveal hidden structure
Suppose overall:
Treatment A appears better than Treatment B.
But after conditioning on disease severity:
P(Recovery | A, Mild disease)
and:
P(Recovery | B, Mild disease)
tell one story,
while severe cases tell another.
Aggregated data may hide subgroup relationships.
This connects to Simpson's paradox.
Conditioning is not automatically causal adjustment
This requires care.
It may seem that adding more variables to a conditional model always improves causal understanding.
Not necessarily.
Conditioning on the wrong variable can:
- introduce bias,
- block causal pathways,
- create artificial relationships.
Conditional probability is a mathematical tool.
Causal interpretation requires assumptions about how the variables relate.
Selection is a kind of conditioning
Suppose we analyse:
only university students.
We are effectively conditioning on:
Admitted = true.
Any relationships we observe are now conditional on having passed the admissions process.
As we saw earlier, that selection can create patterns that do not exist in the wider population.
So conditioning can change relationships dramatically.
The reference group matters
Suppose:
20% of all customers cancel.
But among customers who experienced an outage:
60% cancel.
Then:
P(Cancel) = 0.20
while:
P(Cancel | Outage) = 0.60.
Both statistics are correct.
They describe different populations.
Whenever you see a probability, ask:
Conditional on what?
Probabilities are almost always conditional on something
Even probabilities written without an explicit condition rely on background information.
Suppose we say:
P(rain tomorrow) = 40%.
That estimate already depends on:
- today's weather,
- location,
- season,
- available forecasts,
- model assumptions.
A more complete expression might be:
P(rain tomorrow | everything currently known).
We usually omit the condition for convenience.
But it is there conceptually.
AI predictions are conditional on their inputs
Suppose an AI says:
82% probability of default.
A useful question is:
Based on what information?
Perhaps it conditions on:
- income,
- debt,
- repayment history,
- location.
Change the features and the probability may change.
This reminds us that model outputs are not universal properties of a person.
They are conditional predictions based on a particular representation.
Different information produces different probabilities
Two models might evaluate the same person.
Model A knows:
- income,
- debt.
Model B knows:
- income,
- debt,
- repayment history,
- recent employment change.
Their estimated default probabilities may differ.
Neither probability exists independently of the information used to produce it.
Conditional probability and fairness
Suppose we compare:
P(Loan approved | Group A)
with:
P(Loan approved | Group B).
A difference may indicate unequal outcomes.
But perhaps we also want to examine:
P(Loan approved | income, credit history, Group A)
versus:
P(Loan approved | income, credit history, Group B).
Different fairness definitions condition on different information.
This is one reason fairness becomes mathematically and philosophically difficult.
What should be conditioned on?
This is not purely a technical question.
Suppose two students have:
- identical exam performance,
- different access to educational resources.
Should a university consider only exam results?
Should it condition on socioeconomic context?
Different answers encode different concepts of fairness.
Conditional probability can represent the relationships.
It cannot decide which conditions society ought to regard as legitimate.
Conditional probabilities can change through time
Suppose:
P(machine failure in next hour | current state)
is:
2%.
Ten minutes later, the machine temperature rises.
Now:
P(failure | new state)
may become:
15%.
The model continuously updates as state changes.
This is how conditional probability becomes part of real-time intelligent systems.
The state becomes the condition
For many dynamic systems, prediction takes the form:
P(next state | current state, action).
This is extremely important.
The future depends on:
- where the system is now,
- what action is taken.
For example:
P(vehicle position at t+1 | position at t, velocity, steering action).
This is the mathematical foundation of many sequential decision systems.
Actions change conditional distributions
Suppose a patient may receive:
Treatment A
or:
Treatment B.
We may want:
P(Recovery | Current state, Treatment A)
and:
P(Recovery | Current state, Treatment B).
These are different conditional future distributions.
The decision compares them.
So:
STATE + ACTION
↓
CONDITIONAL FUTURE DISTRIBUTION
↓
DECISION
This connects probability directly to causal reasoning and optimisation.
Conditional expected values
We can also calculate expectations conditional on information.
For example:
E[Demand | Temperature = 0°C]
or:
E[Journey time | Rain, Monday, 08:00].
This gives a context-specific expected value.
Many regression models effectively estimate conditional expectations.
Conditional variance
Context can change uncertainty as well as the expected value.
Suppose journey time at:
03:00
is:
- expected 20 minutes,
- very low variance.
At:
08:30
it may be:
- expected 40 minutes,
- very high variance.
So we may care about:
Var(Journey time | Time of day).
The condition changes both:
centre
and:
spread.
Some information reduces uncertainty more than others
Suppose we are predicting electricity demand.
Knowing:
tomorrow is Tuesday
may help slightly.
Knowing:
tomorrow's temperature
may help substantially.
Knowing:
today's shoe sales
may help almost not at all.
Different variables provide different amounts of information about the target.
Conditional distributions make this visible.
Information can be redundant
Suppose we know:
exact geographic coordinates.
Learning the person's city may add little additional information.
The city is largely implied by the coordinates.
Likewise, two highly correlated sensors may provide overlapping evidence.
More features do not always mean proportionally more information.
Information can be misleading
Suppose a feature correlates strongly with an outcome historically.
Conditioning on it can improve historical predictions.
But if the relationship changes in the future, the conditional model may become unreliable.
Again:
historical conditional probability
does not guarantee:
future conditional probability.
Distribution shift remains a challenge.
Conditional probability and language models
Large language models provide a particularly clear example.
Define:
W = next token.
The model estimates:
P(W | context).
The condition is the sequence of tokens already available.
For example:
The capital of France is
produces one distribution.
But:
The largest city in France is
produces another.
Change the context and the next-token probabilities change.
Each generated token changes the condition
Suppose the model selects a token.
That token becomes part of the context.
So:
P(W₁ | Context)
produces the first token.
Then:
P(W₂ | Context, W₁).
Then:
P(W₃ | Context, W₁, W₂).
Generation is therefore repeated conditional prediction.
Context is a temporary state
This provides a useful bridge to later modules.
An LLM's context acts like a temporary informational state.
It contains information relevant to predicting what comes next.
The model repeatedly performs something conceptually like:
CURRENT CONTEXT
↓
CONDITIONAL DISTRIBUTION OVER NEXT TOKENS
↓
SELECT TOKEN
↓
NEW CONTEXT
The loop continues.
Conditional probability and agents
An AI agent may have a much richer state than text context.
Its condition might include:
- current observations,
- memory,
- goals,
- environment,
- previous actions.
Then it might estimate:
P(future state | current state, action).
That conditional model helps the agent reason about what could happen next.
Prediction is always "given what?"
This may be the most important habit to develop.
Whenever someone presents a prediction, ask:
Given what information?
A forecast is not simply:
P(Y).
It is usually closer to:
P(Y | available information).
And the quality of the forecast depends on:
- what information is available,
- whether it is relevant,
- whether relationships remain stable.
The same observation means different things in different contexts
Suppose electricity demand is:
40 GW.
Is that high?
If it is:
03:00 in summer
perhaps yes.
If it is:
18:00 during a cold winter evening
perhaps not.
Values do not exist in isolation.
Conditioning provides the context needed to interpret them.
Conditional probability and surprise
Suppose:
P(Event) = 1%.
The event occurs.
That may be surprising.
But perhaps we then learn:
Condition B was present.
And:
P(Event | B) = 80%.
Given the new context, the event is no longer surprising.
Surprise depends on what information we condition on.
A model can appear wrong because context was missing
Suppose average demand is:
30 GW.
One day it reaches:
50 GW.
That looks unusual.
Then we learn:
- temperature was extremely low,
- several industrial loads were active.
Conditioned on those circumstances, 50 GW may have been entirely plausible.
Good models use relevant context to distinguish:
unexpected
from:
expected given the situation.
Conditional probability is the bridge to Bayes
Conditional probability allows us to express:
P(Evidence | Hypothesis)
and:
P(Hypothesis | Evidence).
These are different.
Bayes' theorem gives us a principled way to connect them.
Conceptually:
WHAT DID WE BELIEVE BEFORE?
HOW LIKELY IS THIS EVIDENCE UNDER EACH POSSIBILITY?
↓
WHAT SHOULD WE BELIEVE NOW?
That is the subject of a later lesson.
Conditional probability in the course framework
We can now refine our recurring model:
PAST OBSERVATIONS
PRESENT CONDITIONS
↓
CONDITIONING INFORMATION
↓
POSSIBLE FUTURES
↓
P(FUTURE | CURRENT INFORMATION)
↓
DECISION
↓
ACTION
↓
NEW INFORMATION
↓
UPDATED CONDITIONAL PROBABILITIES
Prediction is not a static number.
It is a belief that changes as information changes.
The intelligent loop
A dynamic intelligent system repeatedly performs:
OBSERVE
↓
CONDITION ON NEW INFORMATION
↓
UPDATE BELIEFS
↓
PREDICT
↓
DECIDE
↓
ACT
↓
OBSERVE AGAIN
Conditional probability is therefore not just a classroom probability rule.
It is one of the foundations of machines that continuously learn about and respond to their environment.
Ask what the probability is conditional on
Whenever you encounter a probability or prediction, ask:
- What is the event?
- What is the condition?
- What information is known?
- What population are we restricting attention to?
- Is P(A | B) being confused with P(B | A)?
- What is the base rate?
- Are the variables independent?
- Does the condition change the expected value?
- Does it change the variance?
- Is the model conditioning on relevant information?
- Could important context be missing?
- Could the relationship change in the future?
- Are we interpreting a predictive conditional relationship as causal?
These questions prevent many common errors.
The central idea
Conditional probability formalises a simple but profound fact:
What we should believe depends on what we know.
Instead of asking only:
P(A)
we ask:
P(A | B).
New information changes the set of possibilities we consider plausible.
This gives us a mathematical version of learning:
PRIOR UNCERTAINTY
↓
NEW INFORMATION
↓
CONDITIONAL PROBABILITY
↓
UPDATED BELIEF
The same principle appears everywhere:
P(Disease | Test result)
P(Demand | Weather)
P(Failure | Sensor readings)
P(Pedestrian | Camera image)
P(Next token | Context)
P(Next state | Current state, Action)
Conditional probability therefore connects:
data
to:
inference
and:
present information
to:
possible futures.
In the next lesson, we will build on this idea through Bayesian reasoning: how prior beliefs and new evidence can be combined systematically to update what we believe about an uncertain world.