Module 2 — Data: turning the world into information
Lesson 13 of 13
When data lies without lying
Data can be completely accurate and still create the wrong impression.
A number can be correct.
A chart can be correctly drawn.
An average can be mathematically valid.
A model can faithfully reproduce the dataset it was given.
And yet the conclusion we take from all of this can still be misleading.
This is one of the most important lessons in data literacy:
Data does not have to be false to mislead us.
The problem may lie in:
- what was measured,
- what was not measured,
- who was included,
- who was excluded,
- how observations were grouped,
- which time period was selected,
- which spatial scale was used,
- which statistic was reported,
- what denominator was chosen,
- what comparison was made,
- what context was omitted.
The numbers can all be true.
The story can still be wrong.
A statistic is a representation
Suppose a company says:
Average salary is €70,000.
That statement may be completely accurate.
But imagine most employees earn between:
€30,000 and €45,000
while one executive earns several hundred thousand euros.
The mean salary may genuinely be €70,000.
But it does not describe what a typical employee earns.
The statistic is correct.
The interpretation may not be.
This is the difference between:
a mathematically correct summary
and:
a useful representation of reality.
Mean, median and distribution tell different stories
Suppose two communities both have an average household income of:
€50,000
In Community A, nearly everyone earns between:
€45,000 and €55,000.
In Community B:
- half earn €20,000,
- half earn €80,000.
The average is identical.
The societies are not.
A single number has removed information about the distribution.
This matters enormously when we discuss:
- income,
- wealth,
- healthcare,
- energy costs,
- education,
- model performance.
Whenever you see an average, ask:
What does the distribution look like underneath it?
Aggregation removes information
Aggregation is often necessary.
We cannot inspect millions of individual observations every time we make a decision.
So we calculate:
- averages,
- totals,
- percentages,
- regional values,
- national values.
But aggregation always removes information.
Suppose national electricity reliability is:
99.9%.
That sounds excellent.
But perhaps most customers experience almost perfect service while one region experiences frequent outages.
The national statistic is correct.
It simply hides where the problem occurs.
This connects to an idea we introduced earlier:
Time and space matter.
Global performance can hide local failure
Suppose an AI system has:
95% accuracy.
That sounds impressive.
But perhaps its performance is:
- Group A: 99%
- Group B: 78%
- Group C: 96%
The overall accuracy is correct.
But it hides a serious performance problem for Group B.
The same can happen across:
- languages,
- locations,
- ages,
- income groups,
- devices,
- operating conditions.
So when someone reports model performance, ask:
95% for whom?
The denominator matters
Suppose a headline says:
Incidents increased by 100%.
That sounds dramatic.
But perhaps incidents increased from:
1 to 2.
The percentage is correct.
The absolute increase is:
one incident.
Neither statement is false.
But they create very different impressions.
Always ask:
100% of what?
Relative and absolute risk
Suppose a medical treatment reduces the probability of an event from:
2% to 1%.
We could say:
Risk was reduced by 50%.
That is true.
We could also say:
Risk fell by one percentage point.
That is also true.
The first description sounds much larger.
The second provides more information about the underlying probability.
Good data communication often requires both.
Counts need context
Suppose City A reports:
1,000 burglaries.
City B reports:
500 burglaries.
Which city has the larger problem?
We do not know yet.
Perhaps:
City A has 10 million residents
while:
City B has 100,000 residents.
Raw counts and rates answer different questions.
We might need:
burglaries per 100,000 residents.
The denominator is part of the data story.
Time windows change conclusions
Suppose revenue is:
| Year | Revenue |
|---|---|
| 2022 | €80m |
| 2023 | €100m |
| 2024 | €120m |
| 2025 | €90m |
| 2026 | €95m |
Compare:
2025 → 2026
and revenue increased.
Compare:
2024 → 2026
and revenue decreased.
Both statements are true.
The difference is the selected time window.
Whenever you see a trend, ask:
Why does the graph start there?
Starting points matter
Suppose a market crashes and then partially recovers.
A chart beginning at the bottom of the crash may show:
spectacular growth.
A chart beginning before the crash may show:
a continuing decline.
Neither necessarily contains false data.
The starting point changes the story.
Temporal aggregation can hide extremes
Suppose electricity prices during several periods are:
€20
€25
€30
€200
€250
€30
€25
€20
The average may appear relatively ordinary.
But that average hides two enormous price spikes.
If we care about:
- system stress,
- consumer exposure,
- peak prices,
- operational decisions,
the average may be a poor representation.
The same issue occurs with:
- traffic,
- pollution,
- hospital waiting times,
- electricity demand,
- rainfall.
> **Aggregation can make a volatile system look stable.**
---
### Monthly averages can hide daily extremes
Suppose average hospital waiting time for a month is:
**30 minutes**.
That sounds reasonable.
But perhaps:
- most days have waiting times around 10 minutes,
- several extreme days have waiting times exceeding four hours.
The monthly average is correct.
But it may not describe the experience of patients arriving on those extreme days.
The appropriate temporal resolution depends on the question we are asking.
If we care about:
**average resource utilisation**
then a monthly average may be useful.
If we care about:
**whether the service can cope during periods of peak demand**
then it may tell us very little.
---
### Spatial aggregation can hide extremes too
Imagine measuring air pollution across a city.
The city-wide average might be within a recommended limit.
But neighbourhoods beside major roads may experience much higher concentrations.
The city average is not false.
It answers:
> What is the average across the city?
It does not answer:
> What pollution does each person experience?
Those are different questions.
This is why spatial resolution matters.
---
### Where you draw the boundary matters
Suppose we calculate income by:
- country,
- county,
- city,
- neighbourhood.
Different patterns become visible at each scale.
A wealthy neighbourhood and a poor neighbourhood may look average when combined.
Change the boundary and inequality suddenly appears.
The underlying people did not change.
Our representation changed.
---
### Categories can create artificial boundaries
Suppose we divide people into:
- 18–34,
- 35–64,
- 65+.
A 34-year-old and a 35-year-old now belong to different categories.
But there may be almost no meaningful difference between them.
Meanwhile, enormous variation within the 35–64 category is compressed into one group.
Categories help us understand complex datasets.
But categories are decisions made by people.
Nature does not necessarily contain the same boundaries.
---
### Binning can create patterns
Suppose a continuous variable is divided into:
**low**
**medium**
**high**.
Depending on where we place the thresholds, the apparent relationship with another variable may change.
The observations remain the same.
The categories change.
This means we should always ask:
> **How were these groups defined?**
---
### Selection changes the story
Suppose a company surveys existing customers and reports:
> 90% of customers are satisfied.
Perhaps that number is perfectly correct.
But perhaps dissatisfied customers were more likely to leave the service.
The survey therefore describes:
**people who remained customers**.
It may not describe:
**everyone who experienced the service**.
This is selection bias.
The data does not need to contain false answers.
The problem lies in who was available to answer.
---
### Survivorship bias
Imagine studying successful businesses to discover what makes companies succeed.
We examine 100 companies that survived for 20 years.
Perhaps they share certain characteristics.
But what about companies that had the same characteristics and failed?
If failed companies are missing from the dataset, we cannot know whether those characteristics actually explain survival.
We are looking only at:
**the survivors**.
This is **survivorship bias**.