Module 4 — Prediction: learning patterns from the past
Lesson 15 of 16
Shapley values: sharing credit between features
A model may make a prediction using many features at once.
Suppose a house-price model predicts:
€600,000.
We know the baseline prediction is:
€400,000.
So the model's inputs have collectively moved the prediction by:
€200,000.
The question is:
How should we divide that €200,000 of predictive contribution among the features?
This is a credit-allocation problem.
And it becomes difficult as soon as features:
- overlap,
- interact,
- substitute for one another.
Shapley values provide a principled answer.
The core idea
The Shapley value comes from cooperative game theory.
Imagine a group of players working together to create some total value.
The question is:
How much of that value should each player receive?
In machine learning, we reinterpret the problem.
Instead of:
players
we have:
features.
Instead of:
coalition value
we have:
the model prediction associated with a set of features.
Then we ask:
How much does each feature contribute to the prediction, on average, across the different coalitions in which it can participate?
Why ordinary feature importance is not enough
Suppose we have two features:
- location,
- floor area.
A house-price model gives:
No features → €400k
Location only → €520k
Floor area only → €530k
Both → €600k
Now try to assign credit.
If we add location first:
location contributes +€120k.
Then floor area adds:
+€80k.
But if we add floor area first:
floor area contributes +€130k.
Then location adds only:
+€70k.
So:
The contribution of a feature depends on what is already known.
This is why feature value is not intrinsic.
It is marginal and relational.
Two possible orderings
With two features there are only two possible orders.
Order 1
Location → Floor area
Start:
€400k
Add location:
€520k
Location contribution:
+€120k
Add floor area:
€600k
Floor-area contribution:
+€80k
Order 2
Floor area → Location
Start:
€400k
Add floor area:
€530k
Floor-area contribution:
+€130k
Add location:
€600k
Location contribution:
+€70k
Average the marginal contribution
Shapley values say:
Do not privilege one ordering.
Average the contribution across all possible orderings.
For location:
(+€120k + €70k) / 2 = +€95k
For floor area:
(+€80k + €130k) / 2 = +€105k
So:
Location Shapley value = +€95k
Floor-area Shapley value = +€105k
Together:
€95k + €105k = €200k
And:
€400k baseline + €200k = €600k final prediction.
The full difference has been allocated.
The coalition idea
Now imagine more than two features.
Suppose the model uses:
- location,
- floor area,
- garden,
- building age.
A coalition is simply some subset of these features.
Examples:
{location}
{floor area, garden}
{location, age, garden}
and so on.
For each feature, we ask:
What does this feature add when it joins each possible coalition that does not already contain it?
That is its marginal contribution to that coalition.
A feature can contribute differently in different coalitions
Suppose:
garden
adds little value when we know nothing else.
Perhaps:
+€5k.
But once we know the house is:
- large,
- suburban,
garden might add:
+€40k.
So there is no single contribution that exists independently of context.
The Shapley value averages across these contexts in a principled way.
Why average across coalitions?
Because no one coalition should automatically define the feature's value.
If we measure contribution only when a feature is added:
first,
we may overstate its value.
If we measure it only when added:
last,
other features may already have captured much of its information.
Shapley values account for both.
A three-feature example
Suppose we have:
A = location
B = floor area
C = garden.
For Feature A, we might examine contributions such as:
A added to nothing
A added after B
A added after C
A added after B + C.
Each marginal contribution may differ.
The Shapley value combines them using weights determined by coalition size.
The formal idea
For a feature i, the Shapley value is conceptually:
average marginal contribution of feature i across all possible coalitions that do not already contain i.
A common mathematical form is:
φᵢ =
Σ over coalitions S not containing i
weight(S) × [v(S ∪ {i}) - v(S)]
where:
**φᵢ** is the Shapley value for feature i,
**S** is a coalition of features that does not contain feature i,
**v(S)** is the value produced by coalition S,
**S ∪ {i}** is the same coalition after feature i has been added,
and:
**v(S ∪ {i}) - v(S)**
is the **marginal contribution** of feature i to that particular coalition.
The weighting term ensures that the different possible coalitions are considered fairly.
The notation may look complicated, but the underlying idea is simple:
> **Measure what the feature adds in every possible context, then combine those contributions fairly.**
---
### What is v(S) in machine learning?
This is one of the important modelling choices.
In cooperative game theory, **v(S)** might represent:
**money earned by a coalition of players**.
In machine learning, it could represent something like:
**the model's expected prediction when only the features in S are known**.
So the Shapley framework itself is general.
The exact explanation depends on how we define the value function.
---
### The baseline
Shapley explanations usually work relative to some baseline.
Suppose:
**baseline prediction = €400k**.
This might represent:
**the expected model prediction before we know the particular house's features**.
Then the feature Shapley values explain how the model moves from:
**€400k**
to:
**€600k**.
---
### Baseline + contributions = prediction
A key property is:
```text
Baseline
+ Shapley value of Feature 1
+ Shapley value of Feature 2
+ ...
= Final model output
This makes the explanation additive.
For example:
```text
Baseline €400k
Location +€95k
Floor area +€105k
------------------------
Prediction €600k
With more features:
Baseline €400k
Location +€100k
Floor area +€70k
Garden +€35k
Building age -€5k
------------------------
Prediction €600k
The feature contributions add back exactly to the difference between the baseline and the final model output.
---
### Negative Shapley values
A feature does not have to increase the prediction.
Suppose:
**old building age**
pushes predicted house value downward.
It might receive:
**-€30k**.
Another feature:
**excellent location**
might contribute:
**+€120k**.
Shapley values can therefore show both:
- upward contributions,
- downward contributions.
Importantly, these are movements relative to the chosen baseline.
---
### Shapley values explain the model
This distinction is essential:
> **A Shapley value tells us how a feature contributes to the model's output under a specified explanation framework.**
It does not automatically tell us:
> What would happen in the real world if we intervened and changed that feature?
That is a causal question.
---
### Predictive attribution is not causal attribution
Suppose postcode contributes:
**+€50k**
to a house-price prediction.
This means:
> The model used postcode in a way that raised the prediction relative to the baseline.
It does not mean:
> If we somehow changed only the postcode, the house would causally gain €50k in value.
The postcode may contain information about:
- location,
- transport access,
- local schools,
- neighbourhood characteristics,
- historical property prices.
The Shapley value tells us how the model used that information.
It does not establish which underlying factors caused the house price.
---
### A medical example
Suppose a model predicts:
**30% risk of hospitalisation**.
The baseline risk is:
**10%**.
A Shapley-style explanation might look like:
Baseline risk 10%
Age +8%
Previous admissions +7%
Medication history +4%
Other features +1%
--------------------------
Predicted risk 30%
This tells us which features pushed the model's prediction upward.
It does not prove that changing age would reduce hospitalisation risk by eight percentage points.
---
### Shapley values are local explanations
One of their most useful applications is explaining:
**one particular prediction**.
For example:
> Why was this house valued at €600,000?
> Why was this transaction classified as suspicious?
> Why did this machine receive a high failure-risk prediction?
> Why did this patient receive this risk score?
This is called **local explainability**.
The explanation describes how the features of one particular observation contributed to its prediction.
---
### From local to global
If we calculate Shapley values for many observations, we can aggregate them.
Suppose we calculate them for:
**100,000 houses**.
For every house we obtain a contribution from:
- location,
- floor area,
- garden,
- building age.
We can then ask:
> Which features tend to have the largest contributions across the whole dataset?
One common approach is to calculate:
**average absolute Shapley value**.
Conceptually:
LOCAL SHAPLEY VALUES
↓
AGGREGATE ACROSS OBSERVATIONS
↓
GLOBAL FEATURE IMPORTANCE
---
### Why use absolute values?
Suppose temperature sometimes:
**increases**
an electricity-demand prediction
and sometimes:
**decreases**
it.
Imagine its contributions across two observations are:
**+5 GW**
and:
**-5 GW**.
The ordinary average is:
**0 GW**.
But it would be wrong to conclude that temperature is unimportant.
It strongly affected both predictions.
The effects simply occurred in opposite directions.
So we can instead examine:
**absolute contribution**.
In this example:
**|+5| = 5**
and:
**|-5| = 5**.
Average absolute contribution:
**5 GW**.
This captures how strongly the feature tends to move predictions regardless of direction.
---
### Direction still matters
Magnitude tells us:
> How strongly does this feature affect predictions?
But we may also want to know:
> In which direction?
For example, an electricity-demand model might learn:
**very cold temperature → higher demand**
**mild temperature → lower demand**
**very hot temperature → higher demand**.
The relationship may not even be linear.
Feature attribution can help reveal these patterns.
---
### SHAP
A widely used family of machine-learning explanation methods is called:
**SHAP**.
The name stands for:
**SHapley Additive exPlanations**.
SHAP applies Shapley-style reasoning to machine-learning predictions.
Its aim is to estimate how much each feature contributes to a model output relative to a baseline.
---
### SHAP is not one single algorithm
Different types of models require different computational approaches.
There are SHAP methods designed for:
- tree-based models,
- neural networks,
- general black-box models.
They differ in how the contributions are calculated or approximated.
But the underlying idea remains:
> **Estimate each feature's marginal contribution across different feature coalitions.**
---
### Why approximation is often necessary
The difficulty is combinatorial.
With:
**n features**
there are:
**2ⁿ possible feature subsets**.
For example:
10 features → 1,024 subsets
20 features → 1,048,576 subsets
30 features → 1,073,741,824 subsets
The number of possible coalitions grows extremely quickly.
Calculating the model output for every coalition can therefore become computationally expensive.
---
### Combinatorial explosion
This is an example of **combinatorial explosion**.
A problem may sound modest:
> We only have 30 features.
But the number of ways those features can be combined exceeds:
**one billion**.
This type of problem appears throughout:
- optimisation,
- scheduling,
- network design,
- game theory,
- artificial intelligence.
We often need clever algorithms or approximations rather than exhaustive enumeration.
---
### The four classic Shapley properties
Why use Shapley values rather than simply inventing another way of dividing credit?
One reason is that the Shapley value satisfies several attractive mathematical properties.
These define a particular notion of **fair allocation**.
---
### Efficiency
All of the value is allocated.
If:
Baseline = €400k
and:
Prediction = €600k
then the features collectively explain:
**€200k**.
Their Shapley values must therefore sum to:
**€200k**.
Conceptually:
TOTAL VALUE CREATED
=
SUM OF ALLOCATED CONTRIBUTIONS
Nothing is left unallocated.
---
### Symmetry
Suppose two features make exactly the same contribution in every possible coalition.
There is no principled reason to reward one more than the other.
The Shapley value therefore gives them:
**equal contribution**.
Identical contributors are treated identically.
---
### Dummy
Suppose a feature never changes the model prediction.
Whether it is present or absent:
**nothing changes**.
Its marginal contribution is always:
**zero**.
The Shapley value therefore assigns it:
**zero credit**.
A feature that contributes nothing receives nothing.
---
### Additivity
Suppose we combine two value-allocation problems.
The allocation for the combined problem should behave consistently with the allocations of the individual problems.
The Shapley value satisfies this property.
This becomes particularly useful when value is produced through several interacting components.
---
### Fair in what sense?
The Shapley value is often described as a:
**fair allocation**.
But the word **fair** needs care.
Here it means:
> **The allocation satisfies a particular set of mathematical axioms about how jointly created value should be divided.**
This is **axiomatic fairness**.
It is not automatically the same as:
**social fairness**.
---
### A fair attribution can explain an unfair model
Suppose a discriminatory model relies heavily on:
- postcode,
- income,
- education.
Shapley values might allocate the model's predictive contributions perfectly according to the mathematical axioms.
The attribution can be mathematically fair.
The underlying service can still produce unfair outcomes.
So we should distinguish:
FAIR ATTRIBUTION
≠
FAIR MODEL
≠
FAIR SERVICE
---
### Shapley values can nevertheless help investigate fairness
Suppose a credit model consistently shows:
**postcode**
making large negative contributions for certain groups.
That does not by itself prove discrimination.
But it gives us something important to investigate.
We might ask whether postcode is acting as a proxy for:
- socioeconomic status,
- ethnicity,
- historical disadvantage.
Explainability makes the model's behaviour more visible.
---
### Explanation does not equal justification
Suppose a model rejects a loan application.
A Shapley explanation says:
> Postcode was responsible for a large downward contribution.
That may explain:
**what the model did**.
It does not answer:
> **Should the model have been allowed to use postcode in the first place?**
Explanation and justification are different questions.
---
### The characteristic function matters
There is another deeper issue.
Recall that Shapley theory uses:
**v(S)**
to represent the value produced by coalition S.
The Shapley calculation tells us how to allocate that value.
But someone still has to define:
> **What counts as value?**
This is extremely important.
---
### Shapley theory allocates value; it does not define value
Conceptually:
DEFINE WHAT VALUE MEANS
↓
CALCULATE THE VALUE OF EACH COALITION
↓
APPLY SHAPLEY THEORY
↓
ALLOCATE THE VALUE
The Shapley mechanism solves the allocation problem.
It does not solve the first problem.
---
### This connects directly to loss functions
Earlier we learned:
> **Before a machine can learn, somebody has to define what counts as being wrong.**
Here we encounter a closely related principle:
> **Before value can be allocated, somebody has to define what counts as value.**
The mathematics operates after that decision has been made.
---
### A bad value function still produces a mathematically valid allocation
Suppose we define a completely inappropriate characteristic function.
The Shapley calculation can still allocate its value perfectly according to the axioms.
The mathematics cannot tell us whether:
**the thing being valued was sensible in the first place**.
That remains a modelling and governance question.
---
### Complements
Shapley values become especially interesting when features interact.
Suppose:
**Feature A alone → 0**
**Feature B alone → 0**
but:
**A + B → 100**.
Neither feature creates value alone.
Together they create:
**100**.
They are **complements**.
If the features are symmetric, the Shapley allocation gives:
A = 50
B = 50
The jointly created value is shared between them.
---
### Substitutes
Now consider:
**Feature A alone → 100**
**Feature B alone → 100**
**A + B → 100**.
Either feature can provide all the required information.
Having both creates no additional value.
They are **substitutes**.
Again, if they are symmetric:
A = 50
B = 50
The available credit is shared because either feature could have provided the information.
---
### Why this matters
Features are rarely completely independent contributors.
They may:
- overlap,
- substitute,
- complement,
- interact.
So simply asking:
> How much is Feature A worth?
may not make sense without asking:
> **Worth given what else is available?**
This is the deeper idea behind Shapley values.
---
### The value of information is relational
Suppose we have no weather information.
An accurate temperature forecast may be extremely valuable.
Now suppose we already have:
- five excellent forecasts,
- local temperature sensors,
- satellite observations.
One additional forecast may contribute almost nothing.
The forecast itself has not changed.
What changed is:
**the coalition around it**.
---
### Information has marginal value
This gives us a broader principle:
> **The value of information depends on what we already know.**
This connects directly to:
- conditional probability,
- Bayesian reasoning,
- feature importance.
New information matters because of:
**how much it changes our current understanding**.
---
### This idea extends beyond machine learning
Replace:
**feature**
with:
**resource**.
Now ask:
> How much does this resource contribute given all the other resources already available?
The mathematical structure is similar.
This is why Shapley values appear far beyond machine-learning explainability.
They can be used to think about:
- cost allocation,
- infrastructure,
- shared services,
- energy systems,
- cooperative production.
---
### From features to generators
Suppose an electricity system has several generators.
One additional generator may contribute very little when:
**generation capacity is abundant**.
The same generator may become extremely valuable when:
**the system is close to scarcity**.
Its value depends on:
**what other resources are available**.
That is structurally similar to the feature-attribution problem.
---
### From features to hospital resources
Suppose a hospital has:
- doctors,
- beds,
- scanners,
- operating theatres.
An additional scanner may provide little value if:
**the limiting resource is operating-theatre capacity**.
But it may become extremely valuable after theatre capacity expands.
Again:
> **The value of one resource depends on the surrounding system.**
---
### From features to compute
Suppose an AI service uses:
**100 servers**.
Adding one more server during:
**low demand**
may create almost no additional service value.
During:
**extreme peak demand**
it might prevent requests from being dropped.
Its marginal value depends on:
**system state**.
---
### A broader systems principle
Shapley theory exposes an idea that will return throughout this course:
> **Value is often not an intrinsic property of an individual component.**
Instead:
**VALUE**
emerges from:
**MARGINAL CONTRIBUTION TO A SYSTEM**.
This is true for:
- information,
- infrastructure,
- resources,
- agents.
---
### The central idea
Shapley values solve a deceptively simple problem:
> **When several features jointly create a prediction, how should we share credit among them?**
The difficulty is that feature contributions depend on:
**what other features are already available**.
Shapley theory handles this by considering:
**different possible coalitions**
and calculating:
**marginal contribution across them**.
Conceptually:
BASELINE
↓
FEATURE COALITIONS
↓
MARGINAL CONTRIBUTIONS
↓
FAIR ALLOCATION
↓
SHAPLEY VALUES
↓
BASELINE + CONTRIBUTIONS = PREDICTION
The result is a principled way of sharing predictive credit between interacting features.
But the lesson is bigger than machine learning.
> **A component's value is often determined not by what it can do alone, but by what it contributes to the system given everything else already present.**
That idea will become increasingly important as we move from:
**prediction**
to:
**decision-making**
to:
**allocation of shared resources**.