Enleashed

5.13 Distribution Shift

Imagine we train a model to predict electricity demand.

For years, the relationship between temperature and electricity use is fairly stable.

Cold weather increases heating demand.

Warm weather has a smaller effect.

The model learns from this history and performs extremely well.

Then something changes.

Millions of households install heat pumps.

Millions of vehicles become electric.

Industrial processes electrify.

The same temperature now produces a completely different pattern of electricity demand.

The model has not changed.

The code is identical.

The parameters are identical.

The input data is still being processed correctly.

But the relationship between the world and the model has changed.

This is distribution shift.

And it exposes one of the most important limitations of learning from the past:

A model can only learn relationships contained in the world that generated its training data.

If the world changes, those relationships may no longer hold.

The hidden assumption behind machine learning

When we train a machine-learning model, we usually have historical data:

[ (X_{\text{train}},Y_{\text{train}}) ]

and hope the model will perform well on future data:

[ (X_{\text{future}},Y_{\text{future}}) ]

Hidden inside that hope is an enormous assumption:

The future will resemble the past sufficiently for the relationships learned from historical data to remain useful.

In statistical terms, we are hoping that the distribution generating future observations is sufficiently similar to the distribution that generated the training data.

Very loosely:

[ P_{\text{train}}(X,Y) \approx P_{\text{future}}(X,Y) ]

But sometimes:

[ P_{\text{train}}(X,Y) \neq P_{\text{future}}(X,Y) ]

The distribution has shifted.

The model is now operating in a world different from the one it learned.

What is a distribution?

Earlier in the course we introduced probability distributions.

A distribution describes how possible values are arranged and how likely they are.

For example, suppose household electricity demand historically looked approximately like:

[ P_{\text{old}}(D) ]

After widespread adoption of EVs and heat pumps, demand might instead follow:

[ P_{\text{new}}(D) ]

Perhaps:

average demand increases,
evening peaks become larger,
weather sensitivity changes,
overnight demand increases,
variation between households changes.

The data distribution itself has changed.

That may be enough to reduce model performance.

But there are several different ways distribution shift can occur.

Covariate shift

Suppose our model predicts:

[ Y ]

using inputs:

[ X ]

Covariate shift occurs when the distribution of the inputs changes:

[ P_{\text{train}}(X) \neq P_{\text{deploy}}(X) ]

while the relationship between inputs and outcomes remains approximately stable:

[ P(Y\mid X) ]

does not fundamentally change.

Imagine a medical model trained mainly on patients aged:

[ 30-60 ]

but later deployed in a population containing many more patients over:

[ 80 ]

The relationship between particular clinical measurements and disease may still be similar.

But the system is now seeing a different mixture of inputs.

The deployment population no longer resembles the training population.

A simple example

Suppose a model predicts energy consumption using outdoor temperature.

During training, almost all observed temperatures lie between:

[ 0^\circ C ]

and:

[ 25^\circ C ]

Then the model is deployed during an extreme heatwave:

[ 40^\circ C ]

It is now being asked to make predictions in a region of the input space that was barely represented in its training data.

Even if the underlying relationship has not fundamentally changed, the model is being pushed outside the environment in which it learned.

This is sometimes called out-of-distribution prediction.

And it should make us cautious.

Label shift

Another possibility is that the frequency of outcomes changes.

Suppose an AI model predicts whether a transaction is fraudulent.

During training:

[ P(\text{fraud})=0.01 ]

One transaction in 100 is fraudulent.

Later, a new form of fraud becomes widespread:

[ P(\text{fraud})=0.05 ]

Now one in twenty transactions is fraudulent.

The prevalence of the target outcome has changed.

This is sometimes called label shift or prior probability shift.

Even if many relationships between inputs and outcomes remain similar, probabilities estimated under the old environment may now be badly calibrated.

Concept drift

A more fundamental problem occurs when the relationship itself changes.

During training:

[ P_{\text{old}}(Y\mid X) ]

describes how inputs relate to outcomes.

Later:

[ P_{\text{new}}(Y\mid X) ]

is different.

This is often called concept drift.

Imagine a model predicting whether somebody will click on an online advertisement.

Historically, a particular set of features strongly predicts clicks.

Then people's behaviour changes.

Perhaps:

users become accustomed to advertisements,
a new platform changes interaction patterns,
regulations change tracking,
a new generation of users behaves differently.

The same input no longer implies the same probability of the outcome.

The concept the model learned has moved.

The world can change without warning

Distribution shift does not always happen gradually.

Sometimes the world changes abruptly.

Consider:

a pandemic,
a financial crisis,
a war,
a natural disaster,
a major regulatory change,
a sudden technology breakthrough.

A model trained on years of apparently stable historical data may suddenly become unreliable.

Imagine a model forecasting public transport demand in early 2020.

It may have learned extremely stable relationships from years of commuting behaviour.

Then widespread lockdowns occur.

The historical data has not become mathematically incorrect.

It has simply become a poor description of the new world.

Structural breaks

Economists and time-series analysts often talk about structural breaks.

A structural break occurs when the process generating the data changes substantially.

Before the break:

[ Y_t=f_{\text{old}}(X_t) ]

After the break:

[ Y_t=f_{\text{new}}(X_t) ]

For example, an electricity market might operate under one regulatory structure for years.

Then market rules change.

Behaviour changes.

Prices respond differently.

Historical relationships learned under the old rules may no longer describe the new system.

The same is true when technologies or institutions change.

The model may be describing a regime that no longer exists.

Slow drift

Not every shift is dramatic.

Sometimes the world changes slowly.

Suppose a model predicts residential electricity consumption.

Over ten years:

LED lighting becomes widespread,
insulation improves,
appliances become more efficient,
heat pumps appear,
EV adoption grows,
working-from-home patterns change.

There may be no single moment when the model suddenly becomes wrong.

Instead:

[ P_t(X,Y) ]

gradually evolves.

A model can therefore decay slowly.

Performance today may be slightly worse than last year.

Next year slightly worse again.

Eventually the model may become seriously unreliable.

This is why deployed models need monitoring.

Seasonal change is not necessarily distribution shift

Some changes are expected.

Electricity demand differs between:

summer,
winter,
weekdays,
weekends.

Retail demand changes around Christmas.

Traffic changes throughout the day.

If the model has learned those recurring patterns, they are part of the expected data-generating process.

Distribution shift usually refers to a change that makes the deployment environment meaningfully different from what the model learned.

The distinction is between:

[ \text{known variation} ]

and:

[ \text{changing system} ]

A good model may cope very well with seasonality while failing badly after a structural transformation.

Climate change creates distribution shift

Consider a flood prediction model trained on decades of historical rainfall.

Historically:

[ P_{\text{historical}}(\text{extreme rainfall}) ]

may have been very small.

But if the climate changes, the probability distribution governing future rainfall may differ:

[ P_{\text{future}}(\text{rainfall}) \neq P_{\text{historical}}(\text{rainfall}) ]

A model built on the assumption that past frequencies remain constant may systematically underestimate future risk.

This reveals a deeper problem.

Historical data is valuable because it tells us what happened.

But sometimes the very reason we are interested in predicting the future is because we expect the future to be different.

Technological transitions create distribution shift

Imagine predicting road fuel demand using fifty years of historical data.

The model may learn relationships involving:

population,
income,
fuel prices,
vehicle ownership.

Then electric vehicles become dominant.

The historical relationship between:

[ \text{vehicle kilometres} ]

and:

[ \text{petrol demand} ]

breaks.

No amount of beautifully fitted historical regression can solve the fundamental problem:

The underlying technology changed.

The same can happen with:

renewable energy,
batteries,
AI,
automation,
teleworking,
robotics,
biotechnology.

Past behaviour is not guaranteed to survive technological transformation.

Policy creates distribution shift

Suppose we train a model predicting household energy consumption under one tariff structure.

Consumers historically face:

[ \text{flat electricity prices} ]

Then a new system introduces highly dynamic prices.

Households install automated devices that respond to those prices.

The relationship:

[ P(\text{demand}\mid\text{temperature,time}) ]

may change because demand is now responding to an entirely new signal.

The policy has changed the behaviour the model was trying to predict.

This gives us an important insight:

Changing the rules of a system can invalidate models trained under the old rules.

Prediction itself can create distribution shift

AI systems can create an even stranger problem.

Suppose a recommendation algorithm predicts what people are likely to watch.

It recommends those items.

People watch them.

The next training dataset therefore contains behaviour partly created by the previous model.

We have:

[ \text{historical behaviour} \rightarrow \text{model} \rightarrow \text{recommendations} \rightarrow \text{new behaviour} \rightarrow \text{new training data} ]

The model changes the distribution from which its future data comes.

This is sometimes called performative prediction.

The prediction does not merely describe the world.

It helps create the world it will later observe.

A credit model can change its own population

Suppose a bank uses an AI model to decide who receives loans.

The model rejects applicants considered high risk.

Only accepted applicants subsequently generate repayment histories.

The bank's future dataset therefore contains outcomes primarily for people the earlier model approved.

The dataset is no longer a neutral sample of the original applicant population.

It has been filtered by previous decisions.

This creates:

[ \text{prediction} \rightarrow \text{selection} \rightarrow \text{observed outcomes} \rightarrow \text{training data} ]

The model influences which parts of reality become visible.

Distribution shift and feedback loops

Consider predictive policing.

Historical data suggests more crime in Area A.

The system sends more police to Area A.

More police observe more offences.

Recorded crime increases.

The model is retrained.

Area A again appears high risk.

The data distribution is changing partly because the prediction altered observation.

The loop becomes:

[ \text{data} \rightarrow \text{prediction} \rightarrow \text{allocation} \rightarrow \text{measurement} \rightarrow \text{new data} ]

Distribution shift is therefore not always an external disturbance.

Sometimes it is endogenous.

The system creates it.

Training distribution versus deployment distribution

A crucial question in machine learning is:

Where did the training data come from?

Suppose an image-recognition model is trained using high-quality photographs taken in daylight.

It is then deployed on:

CCTV footage,
low-light images,
unusual camera angles,
poor-quality sensors.

The underlying task may be identical.

But:

[ P_{\text{train}}(X) \neq P_{\text{deploy}}(X) ]

The model may fail because the visual environment has changed.

A benchmark measured under one distribution does not guarantee performance under another.

The laboratory is not the world

This problem appears frequently when AI systems leave controlled experiments.

A model might perform exceptionally on a carefully constructed dataset.

Then deployment introduces:

different users,
different sensors,
different geography,
different behaviour,
missing data,
unexpected edge cases,
changing incentives.

Performance falls.

Nothing mysterious happened.

The deployment environment was not sampled from exactly the same distribution as the benchmark environment.

This is why:

[ \text{test performance} ]

is not the same thing as:

[ \text{guaranteed real-world performance} ]

Geography creates distribution shift too

Suppose an AI healthcare model is trained in London.

It is deployed in another country.

Patient populations may differ.

Disease prevalence may differ.

Healthcare systems may differ.

Measurement practices may differ.

The relationship between variables may therefore change.

Formally:

[ P_{\text{London}}(X,Y) \neq P_{\text{deployment}}(X,Y) ]

This connects directly to our previous lesson on spatial prediction.

A model that generalises across time may still fail across space.

A model that generalises across space may fail over time.

Real-world deployment often requires both.

Measurement systems can change

Sometimes the world remains similar but the way we measure it changes.

Suppose a hospital replaces one diagnostic instrument with a newer model.

The new device systematically records slightly different values.

The patient's biology has not changed.

But:

[ P(X) ]

has.

A machine-learning model consuming those measurements may suddenly behave differently.

Similar effects can occur when:

sensors are recalibrated,
database definitions change,
questionnaires change,
accounting rules change,
data pipelines change.

Distribution shift can therefore originate in the measurement system, not merely the underlying reality.

Labels can change meaning

Imagine a historical dataset where:

[ Y=1 ]

means:

customer churned within 90 days.

Later, the organisation changes the definition to:

customer cancelled their subscription.

Those sound similar.

They may not be equivalent.

Perhaps customers can now pause accounts.

Perhaps dormant accounts are treated differently.

Perhaps business rules changed.

If the meaning of the label changes, comparing old and new observations becomes dangerous.

The variable may have the same name.

The underlying concept may not.

Models can fail precisely when they matter most

Suppose a model predicts electricity demand extremely accurately under normal conditions.

Its average error is:

[ 1.5% ]

Excellent.

But during extreme cold weather, its error rises to:

[ 15% ]

Those extreme periods may be exactly when:

network capacity is scarce,
electricity prices are high,
security margins are small.

Average performance may therefore hide catastrophic conditional performance.

Distribution shift often exposes models at precisely the unusual moments when decisions become most consequential.

This is one reason rare events deserve particular attention.

Out-of-distribution detection

Ideally, a model should sometimes recognise:

This situation is unlike the data I was trained on.

This is the goal of out-of-distribution detection.

Conceptually, we might ask whether a new input:

[ x^* ]

appears plausible under the training distribution:

[ P_{\text{train}}(X) ]

If:

[ P_{\text{train}}(x^*) ]

is extremely low, the system may flag the input as unfamiliar.

Rather than confidently making a prediction, it could say:

This situation is outside the environment in which I have reliable experience.

That can be enormously valuable.

Knowing when you do not know

Consider two AI systems confronted with genuinely novel conditions.

System B

Current conditions differ substantially from the data on which this model was trained. Forecast uncertainty is unusually high.

Which is more intelligent?

The first appears more decisive.

The second may be far safer.

A system's ability to recognise the limits of its own experience is an important part of trustworthy prediction.

Intelligence should not merely mean producing an answer.

Sometimes it should mean recognising:

I have not seen enough situations like this to justify confidence.

Monitoring deployed models

A model should therefore not simply be trained, deployed and forgotten.

We can monitor whether inputs are changing.

For example, compare:

[ P_{\text{train}}(X) ]

with:

[ P_{\text{current}}(X) ]

We can examine changes in:

means,
variances,
category frequencies,
correlations,
prediction distributions,
error rates.

If labelled outcomes eventually become available, we can also track whether:

[ P(Y\mid X) ]

appears to be changing.

This turns machine learning into an ongoing operational process rather than a one-off modelling exercise.

Performance monitoring

Suppose a forecasting system historically achieves:

[ MAE=2.1 ]

Over successive months:

[ 2.2 ]

[ 2.5 ]

[ 3.1 ]

[ 4.0 ]

Something is changing.

Perhaps:

the population changed,
behaviour changed,
sensors changed,
the environment changed,
the model is no longer appropriate.

Performance monitoring can reveal this deterioration.

But only if we continue measuring what happens after deployment.

Retraining

One response to distribution shift is to retrain the model using newer data.

Suppose the model was originally trained on:

[ 2015-2020 ]

By 2026, we might retrain using:

[ 2021-2026 ]

This allows the model to learn more recent relationships.

But retraining is not magic.

If a fundamental structural change is underway, even recent data may poorly describe tomorrow.

And if a model retrains automatically on data influenced by its own previous decisions, feedback loops may become embedded rather than corrected.

Retraining must therefore be treated as part of system design.

How much history should we keep?

Suppose we have twenty years of data.

Older data gives us:

more observations,
more rare events,
more examples of unusual conditions.

But older data may describe:

obsolete technologies,
old behaviours,
previous regulations,
different populations.

Recent data may be more relevant but less diverse.

There is therefore a trade-off:

[ \text{more history} ]

versus:

[ \text{more relevance} ]

Sometimes we may weight recent observations more strongly.

Sometimes we may use rolling windows.

Sometimes historical extremes are too valuable to discard.

There is no universal solution.

Adaptation

Some models are designed to adapt continuously.

As new observations arrive:

[ (X_{t+1},Y_{t+1}) ]

the model updates.

This can help in environments that change gradually.

We might imagine:

[ \theta_t \rightarrow \theta_{t+1} ]

where (\theta) represents the model parameters.

The model evolves with the system.

But adaptation introduces new questions.

How quickly should it forget?

What if recent data is anomalous?

What if someone deliberately manipulates the inputs?

What if rapid adaptation destroys useful long-term knowledge?

Adaptive systems require their own safeguards.

Robustness

Another strategy is to design models that perform reasonably across a range of possible distributions.

Rather than optimising only for:

[ P_{\text{train}} ]

we may ask whether the model performs well under plausible variations.

This is part of robust machine learning.

The objective shifts from:

Be extremely good under one assumed world.

towards:

Remain useful across a range of worlds that might reasonably occur.

This can be especially important in high-stakes systems.

Stress testing

Banks do not merely ask:

What happens under normal economic conditions?

They may ask:

What happens under a severe recession?

Electricity systems ask:

What happens during extreme demand, generator failure or network outages?

AI systems can be examined similarly.

Instead of evaluating only average historical performance, we can deliberately test:

unusual inputs,
extreme conditions,
missing information,
altered populations,
changed correlations.

This is stress testing.

It asks:

How does the system behave when reality stops looking normal?

Scenario analysis

For long-term decisions, we may not know the future distribution at all.

Consider predicting electricity demand in 2050.

We do not know:

future technologies,
policy,
prices,
population,
industrial structure,
consumer behaviour.

Rather than pretending we know one probability distribution, we might construct scenarios:

Scenario D

High energy efficiency.

Now we are testing decisions across multiple plausible futures.

The objective may become less about predicting exactly which future occurs and more about designing decisions that remain sensible under several futures.

Prediction versus robustness

This distinction is important.

Sometimes the natural question is:

What is most likely to happen?

But for important decisions another question may be better:

What decision performs acceptably across the range of things that could happen?

These are different philosophies.

The first focuses on prediction.

The second focuses on robustness.

When uncertainty about the future is large, a slightly less optimal decision that remains safe across many possible futures may be preferable to a highly optimised decision that collapses if one assumption changes.

Distribution shift and fairness

Distribution shift can also affect groups differently.

Suppose a model performs equally well across two populations during training.

Later, behaviour changes primarily in one group.

Now:

[ \text{error}_A ]

may remain low while:

[ \text{error}_B ]

increases dramatically.

Overall accuracy might fall only slightly because Group B is relatively small.

But the consequences for Group B could be severe.

Monitoring only global performance can therefore hide unequal model degradation.

This connects distribution shift to fairness.

We may need to ask:

Whose distribution shifted?

not merely:

Did the overall distribution shift?

Historical injustice can persist through shift

There is another complication.

Suppose historical data contains an unjust pattern.

Then society changes.

The historical relationship may begin weakening.

A model trained on the old world may continue reproducing the old pattern after reality has started changing.

For example, historical hiring data may reflect a period in which particular groups had fewer opportunities.

If labour markets become more equitable, a model trained heavily on older data could continue predicting according to relationships that no longer represent the desired or emerging reality.

Historical data can therefore act as a kind of inertia.

It can make yesterday's world persist inside tomorrow's algorithm.

Not every historical pattern should be preserved

This is a profound point.

Machine learning often assumes that discovering stable historical relationships is desirable.

But suppose the historical relationship exists because of:

discrimination,
poor policy,
obsolete technology,
institutional failure.

Do we actually want the future to reproduce it?

A model may accurately learn:

[ \text{what happened} ]

without answering:

[ \text{what should happen} ]

Distribution shift can sometimes represent progress.

A model degrading because an unjust historical relationship is disappearing is not necessarily a societal failure.

It may be exactly the future we wanted.

The future may deliberately differ from the past

Consider climate policy.

Suppose we want to predict future fossil-fuel consumption.

Historical data may suggest one trajectory.

But the entire purpose of policy may be to break that historical trajectory.

Likewise:

public-health interventions aim to change disease patterns,
education policy aims to change outcomes,
energy policy aims to change consumption,
transport policy aims to change mobility,
AI itself aims to change productivity.

In these cases:

The future resembling the past may actually represent policy failure.

This creates a fascinating tension.

Machine learning often learns by assuming continuity.

Policy often exists precisely to create discontinuity.

Prediction under intervention

Suppose historical data tells us:

[ P(Y\mid X) ]

But now we introduce an intervention:

[ A ]

The future relationship may become:

[ P(Y\mid X,A) ]

If the intervention has never previously existed, the historical data may contain little direct evidence about the new regime.

We are now moving from simple prediction towards:

causal inference,
simulation,
structural modelling,
experimentation.

This is another reason prediction alone cannot answer every question about the future.

The model does not know history has changed

A trained model does not wake up one morning and realise:

The world has entered a new regime.

Unless we build mechanisms to detect change, it simply receives an input:

[ X ]

and applies the function it learned:

[ \hat{Y}=f(X) ]

The model has no inherent guarantee that:

[ f ]

still describes reality.

This gives us a crucial distinction:

[ \text{model certainty} \neq \text{world stability} ]

A model may produce an extremely confident prediction in a world where its assumptions have stopped being valid.

Large language models and distribution shift

The same principle applies to large language models.

An LLM is trained on information generated before some point in time.

The world continues changing.

New:

laws,
governments,
scientific discoveries,
companies,
products,
events,
terminology

appear.

A model's learned representation of the world may therefore become stale.

This is one reason systems may require access to:

current information,
retrieval systems,
tools,
databases,
updated training.

The knowledge distribution changes while the static model remains fixed.

Agents make the problem harder

Now imagine an AI agent operating continuously in the world.

It:

observes,
predicts,
decides,
acts.

Its actions alter the environment.

The next observations therefore arrive from a state partly created by previous actions.

We now have:

[ P_{t+1}(X,Y) ]

partly depending on:

[ A_t ]

the agent's previous action.

The agent does not merely encounter distribution shift.

It may generate distribution shift.

This is one of the reasons the transition from prediction systems to autonomous systems is so consequential.

Static intelligence in a dynamic world

We can now see the fundamental problem.

Machine learning often begins with:

[ \text{historical dataset} \rightarrow \text{model} ]

The dataset is fixed.

The model may initially be fixed.

But the world is not.

The real system looks more like:

[ P_1 \rightarrow P_2 \rightarrow P_3 \rightarrow \cdots ]

The data-generating distribution evolves continuously.

A model is therefore never simply:

good

or:

bad.

A better question is:

Good for which environment, population, place and period?

The deeper lesson

This entire module has been asking:

When is a prediction good?

Distribution shift reveals why that question can never be answered permanently.

A model might have:

excellent accuracy,
excellent calibration,
excellent scoring-rule performance,
excellent historical validation.

And still fail tomorrow.

Because every evaluation was performed under some distribution:

[ P_{\text{evaluation}}(X,Y) ]

Deployment occurs under another:

[ P_{\text{deployment}}(X,Y) ]

The central question is whether those worlds are sufficiently similar.

So we arrive at one of the most important principles in machine learning:

Past performance is evidence about future performance only to the extent that the future resembles the conditions under which that performance was measured.

This returns us all the way to the beginning of the course.

We started with:

[ \text{PAST} \rightarrow \text{DATA} \rightarrow \text{MODEL} \rightarrow \text{PREDICTION} \rightarrow \text{FUTURE} ]

But now we can see the dangerous assumption hidden inside that sequence:

[ \text{PAST} \approx \text{FUTURE} ]

Sometimes that assumption is reasonable.

Sometimes it is spectacularly wrong.

So whenever somebody presents an AI model with impressive historical performance, ask:

What world generated the training data?

Is that still the world the model is operating in?

Which relationships are assumed to remain stable?

How would we know if they stopped being stable?

What happens when the system encounters something genuinely new?

And perhaps the most important question:

Are we actually trying to predict a future that resembles the past — or are we trying to create a future in which it doesn't?