Enleashed

Data begins with measurement.

But measurements are rarely perfect.

A thermometer might report:

20.3°C

when the true temperature is slightly higher or lower.

A GPS receiver might estimate a location several metres away from the true position.

A smart meter may have a small measurement tolerance.

A person completing a survey may misremember how much they spent last month.

A camera may distort colour because of lighting.

A medical test may produce an imperfect indication of whether a condition is present.

None of this necessarily makes the data useless.

It reflects a basic reality:

Measurement is an imperfect process for learning about the world.

The important question is not whether measurement error exists.

It almost always does.

The important questions are:

how large is it?
where does it come from?
is it random or systematic?
how does it affect what we infer?
does it matter for the decision we need to make?

The measurement is not the thing being measured

Suppose the actual temperature of a room is:

20.37°C

A sensor reports:

20.4°C

The number stored in the database is not the physical temperature itself.

It is an observation produced by a measurement system.

Conceptually:

REAL WORLD

↓

MEASUREMENT SYSTEM

↓

OBSERVATION

The measurement system might include:

a sensor,
electronics,
calibration,
software,
communication networks,
rounding,
data processing.

Each stage can affect the final value.

This gives us an important distinction:

Observation ≠ reality.

An observation is evidence about reality.

A simple model of measurement

We can think of a measurement as:

MEASURED VALUE = TRUE VALUE + ERROR

or mathematically:

y = x + e

where:

x is the true quantity,
y is the observed measurement,
e is measurement error.

If:

x = 20.37°C

and:

e = +0.03°C

then:

y = 20.40°C

This simple equation is useful because it reminds us that the number we observe may differ from the underlying quantity we care about.

But what is the "true" value?

Even the idea of a true value deserves some thought.

Suppose we ask:

What is the temperature of this room?

Where exactly?

At the ceiling?

Near the window?

Beside the radiator?

At the centre of the room?

At what moment?

Temperature varies through:

space,
time.

So before worrying about measurement error, we must define what quantity we are trying to measure.

Perhaps the real question is:

What is the air temperature one metre above the floor at this location and this moment?

Measurement begins with definition.

Precision is not accuracy

Two ideas are often confused:

accuracy

and:

precision.

Accuracy describes how close a measurement is to the quantity we are trying to measure.

Precision describes how consistently a measurement can be repeated.

Imagine measuring the same object several times.

Sensor A

Sensor B

9.51
9.50
9.52
9.51

Suppose the true value is:

10.00

Sensor B is very precise.

Its measurements are tightly grouped.

But they are systematically wrong.

Sensor A is both precise and accurate.

A precise number can still be wrong

Suppose a model receives:

temperature = 21.847392°C

The number looks extremely precise.

But perhaps the sensor is only accurate to:

±0.5°C

The extra decimal places do not create additional knowledge.

This is an important principle:

Numerical precision should not be confused with certainty.

A computer can store many decimal places.

That does not mean the physical measurement supports them.

Resolution is another concept

A sensor may measure only in increments of:

0.1°C

This is its measurement resolution.

So it might report:

20.1

then:

20.2

but never:

20.1537

Higher resolution allows finer distinctions.

But higher resolution does not automatically mean higher accuracy.

A device can produce very detailed numbers that are systematically wrong.

Measurement error comes from many sources

Measurement error can enter through:

sensor limitations,
calibration,
environmental conditions,
sampling frequency,
rounding,
human judgement,
data entry,
communication,
preprocessing.

Different sources of error behave differently.

Understanding the measurement process helps us understand the data.

Sensor noise

Physical sensors often contain noise.

Suppose a stationary temperature sensor reports:

20.1
20.2
20.0
20.1
20.2
20.1

The actual temperature may be almost constant.

Small fluctuations in the measurements arise from the measurement process.

This is normal.

A noisy measurement is not necessarily a bad measurement.

We simply need to understand its uncertainty.

Random error

Some measurement errors vary unpredictably around the true value.

Suppose repeated measurements are:

9.9
10.1
10.0
9.8
10.2

If the errors are roughly balanced around:

10.0

then averaging several measurements may give a good estimate.

This is one advantage of repeated measurement.

Random errors can sometimes average out.

Systematic error

Other errors consistently push measurements in one direction.

Suppose a scale always reports:

0.5 kg too high.

Then:

70 kg → 70.5 kg

80 kg → 80.5 kg

90 kg → 90.5 kg

Taking more measurements does not solve the problem.

The average will still be wrong.

This is a systematic error, or bias.

More data does not fix systematic measurement error

This distinction is extremely important in the age of big data.

Suppose we have:

10 measurements

from a badly calibrated sensor.

Then we collect:

10 million measurements

from the same badly calibrated sensor.

We now know the wrong value extremely precisely.

More observations can reduce random uncertainty.

They cannot automatically correct systematic error.

More data cannot rescue a measurement system that is systematically measuring the wrong thing.

Calibration

Sensors are often calibrated against a trusted reference.

Suppose a temperature sensor is placed beside a high-quality reference instrument.

The reference reports:

20.0°C

Our sensor reports:

20.7°C

We can investigate whether the sensor consistently has an offset.

Calibration attempts to understand and correct the relationship between:

measured value

and:

reference value.

Calibration is therefore part of the measurement system.

Calibration can drift

A sensor that was accurate when installed may become less accurate over time.

Components:

age,
wear,
become contaminated,
respond to environmental conditions.

This is sometimes called sensor drift.

For example:

Year 1 → error ≈ 0.1°C

Year 3 → error ≈ 0.5°C

A measurement system may therefore need periodic recalibration.

The reliability of data can change through time.

Environmental conditions affect measurement

Suppose a temperature sensor is mounted in direct sunlight.

It may report a higher temperature than the surrounding air.

The sensor itself may work perfectly.

The problem is where and how it was installed.

Likewise:

microphones behave differently in noisy environments,
cameras behave differently in darkness,
GPS performs differently inside buildings,
air-quality sensors respond to humidity.

Measurement error often depends on context.

Measurement error can vary through space

Imagine a network of air-quality sensors.

Sensors near roads may experience different conditions from sensors in parks.

Buildings may interfere with airflow.

Different sensor models may be installed in different areas.

So measurement uncertainty can vary geographically.

A map may therefore contain not only:

different measured values

but also:

different levels of measurement confidence.

Measurement error can vary through time

A sensor might perform well during normal conditions but poorly during extremes.

For example:

a sensor may saturate above a maximum value,
batteries may perform poorly in extreme cold,
cameras may struggle at night,
communications may deteriorate during storms.

The measurement process itself can therefore change through time.

Measurement ranges matter

Every sensor has limits.

Suppose a sensor can measure:

0–100 units

What happens if the true value is:

120?

Perhaps the sensor reports:

100

because it has reached its maximum.

The recorded data now contains:

100

but the true interpretation is:

100 or greater.

If we ignore the measurement limit, we may seriously misinterpret the observation.

Saturation

This is sometimes called saturation.

Imagine a camera looking at an extremely bright light.

Pixels may reach their maximum possible value.

Once saturated, they cannot distinguish:

very bright

from:

extremely bright.

Information has been lost.

The measurement system has reached the boundary of what it can represent.

Detection limits

At the other end, some instruments cannot reliably detect extremely small quantities.

A laboratory test might report:

below detection limit

That does not mean:

zero.

It means the quantity is too small for the measurement system to estimate reliably.

Again:

not observed ≠ absent.

Quantisation and rounding

Digital systems often represent continuous quantities using discrete values.

Suppose temperature is stored to the nearest degree.

The true value:

20.4°C

becomes:

20°C

while:

20.6°C

becomes:

21°C.

Information has been lost through rounding.

Usually this may be harmless.

But if a decision boundary occurs at:

20.5°C

the rounding could matter.

Measurement frequency creates error too

Suppose electricity demand changes every second.

But we measure only once every hour.

The hourly measurement may miss:

peaks,
short events,
rapid changes.

The sensor may be perfectly accurate at the moment it measures.

Yet our representation of the underlying process may still be poor.

This is a form of measurement limitation created by sampling through time.

Aliasing

Imagine a process oscillating rapidly.

If we sample too slowly, the measurements can make it appear to behave in a completely different way.

This phenomenon is known as aliasing.

The problem is not inaccurate individual measurements.

The problem is insufficient sampling frequency.

This shows why measurement error is broader than simply asking:

Is the sensor accurate?

We must also ask:

Are we observing the system at an appropriate resolution?

Spatial resolution creates similar problems

Suppose one weather station represents an entire city.

Its temperature measurement may be highly accurate at the sensor location.

But the city may contain:

parks,
dense urban areas,
waterfronts,
hills.

Using one measurement to represent the whole city introduces another kind of error.

The measurement is accurate locally.

The aggregation assumption may not be.

Human measurements contain error too

Not all measurements come from sensors.

Suppose a survey asks:

How many hours did you exercise last month?

A respondent may:

forget,
estimate,
round,
interpret "exercise" differently.

The resulting number is still data.

But its measurement process is human memory and judgement.

That process has uncertainty.

Self-reported data

Many important datasets rely on self-reporting.

Examples include:

income,
diet,
exercise,
alcohol consumption,
mental wellbeing,
political opinions.

People may:

misremember,
misunderstand,
round,
exaggerate,
understate,
choose not to answer truthfully.

The measurement process therefore includes human behaviour.

People sometimes give answers they believe are more socially acceptable.

For example, respondents may underreport behaviour considered undesirable.

Or overreport behaviour considered desirable.

This creates systematic measurement error.

Again, collecting more responses does not necessarily remove the bias.

If the measurement process systematically pushes answers in one direction, the resulting dataset can remain biased at enormous scale.

Labels are measurements too

This is particularly important for machine learning.

Suppose humans label photographs as:

safe,
unsafe.

Or job applications as:

strong,
weak.

Or online comments as:

toxic,
non-toxic.

It is tempting to treat these labels as truth.

But the labels are themselves observations produced by human judgement.

Different people may disagree.

So:

LABEL ≠ PERFECT TRUTH

A label is often another measurement.

Inter-rater disagreement

Suppose five people classify the same comment.

Their labels are:

toxic
toxic
not toxic
toxic
not toxic

What is the "correct" label?

Perhaps there is no perfectly objective answer.

We might use:

majority vote,
expert review,
probabilistic labels.

The disagreement itself contains information about ambiguity.

Collapsing everything to:

toxic = 1

may hide that uncertainty.

Ground truth is not always ground truth

Machine-learning discussions often refer to the correct label as ground truth.

Sometimes that term is reasonable.

For example:

Did this machine fail within 24 hours?

may have a clearly observable answer.

But other labels are much less objective.

Examples include:

creditworthiness,
quality,
attractiveness,
risk,
suitability,
sentiment,
dangerousness.

These may depend partly on definitions and judgement.

We should therefore ask:

How was the ground truth produced?

Proxy measurements

Sometimes we cannot measure the thing we really care about.

So we measure something related to it.

Suppose we want to measure:

student understanding

but use:

exam score.

Or:

employee productivity

using:

number of emails sent.

Or:

health

using:

hospital admissions.

Or:

economic wellbeing

using:

income.

These are proxies.

They may contain useful information.

But they are not identical to the underlying concept.

The proxy can become the target

This creates a dangerous possibility.

Suppose a school wants to improve:

learning.

But performance is measured using:

test scores.

If the institution begins optimising entirely for test scores, behaviour may shift towards improving the metric rather than the underlying goal.

The measurement becomes a target.

This idea is sometimes summarised by Goodhart's Law:

When a measure becomes a target, it can cease to be a good measure.

We will encounter this again when discussing optimisation and service design.

Measurement changes behaviour

Measurement is not always passive.

Suppose employees know their performance is measured by:

calls handled per hour.

They may shorten calls.

Suppose a hospital is evaluated by:

waiting time.

It may change how patients enter the queue.

Suppose social-media creators are rewarded according to:

engagement.

They may produce content designed specifically to maximise engagement.

Once people know the measurement system, they may respond to it.

The measurement system begins influencing the world it measures.

Measurement can create feedback

The loop becomes:

WORLD

↓

MEASUREMENT

↓

METRIC

↓

DECISION / INCENTIVE

↓

BEHAVIOUR CHANGES

↓

NEW WORLD

↓

NEW MEASUREMENT

Measurement therefore becomes part of a feedback system.

This is especially important when AI systems optimise measured outcomes.

Measurement error can propagate into models

Suppose we train a model using:

temperature

but the sensor has measurement noise.

The model is not learning from perfect temperature.

It is learning from:

measured temperature.

Likewise, if labels contain error, the model learns from noisy targets.

Conceptually:

REALITY

↓

MEASUREMENT ERROR

↓

TRAINING DATA

↓

MODEL

Measurement error becomes part of the learning problem.

Errors in inputs

Suppose a model predicts electricity demand using:

temperature,
occupancy,
time.

If temperature measurements are noisy, the model's inputs contain uncertainty.

The model may still perform well.

But there is a limit to how much it can learn from unreliable inputs.

This is sometimes described as errors in variables.

Errors in labels

Suppose a model is trained to identify defective products.

Human inspectors incorrectly label:

5% of examples.

Even a perfect learning algorithm now receives contradictory training evidence.

The model cannot magically know which labels are wrong unless other information helps reveal the pattern.

The quality of labels can therefore place an upper bound on achievable performance.

More complex models do not fix bad measurement

Suppose the real problem is that the target variable is badly measured.

We might try:

a larger neural network,
more layers,
more compute,
more training.

But if the training signal itself is unreliable, greater model complexity may not solve the underlying problem.

Sometimes the most valuable improvement is not:

better AI

but:

better measurement.

Noise can sometimes be handled statistically

Fortunately, measurement error does not mean analysis is impossible.

Suppose sensor noise is:

small,
approximately random,
well understood.

We can often account for it using statistical methods.

Repeated observations can help.

Filtering can help.

Probabilistic models can help.

State estimation can help.

The important thing is to model the uncertainty rather than pretend it does not exist.

Averaging

Suppose a sensor measures the same stable quantity repeatedly:

10.2
9.9
10.1
9.8
10.0

The average is:

10.0

If the noise is approximately random, averaging can reduce its effect.

This is one of the simplest forms of noise reduction.

But remember:

Averaging helps with random error. It does not automatically remove systematic bias.

Filtering

For time-series data, we may use neighbouring observations to reduce noise.

Suppose a sensor reports:

20.1
20.2
25.8
20.3
20.4

The value:

25.8

looks unusual relative to surrounding observations.

Perhaps it is:

a genuine sudden event,
sensor noise,
a fault.

A filter may smooth the signal.

But smoothing introduces assumptions.

We must be careful not to remove genuine events simply because they look unusual.

Outliers are not necessarily errors

An unusual observation may be wrong.

But it may also be the most important observation in the dataset.

Suppose a power system normally operates around:

50 Hz.

A rare frequency excursion may look like an outlier.

But it could indicate a serious system disturbance.

Automatically deleting unusual values can therefore destroy valuable information.

Unusual does not mean incorrect.

Error detection requires context

Suppose a temperature sensor reports:

50°C.

Is that wrong?

In a living room, probably suspicious.

Inside an industrial process, perhaps completely normal.

Data cleaning therefore requires context.

There is no universal threshold separating:

valid

from:

error.

The meaning of a measurement depends on the system.

Physical constraints can help

Sometimes we know what values are physically possible.

For example:

battery state of charge = 140%

is probably invalid.

negative age

is impossible.

vehicle speed = 40,000 km/h

is unlikely for a road vehicle.

Physical and logical constraints can help detect measurement errors.

This is one reason domain knowledge remains important in data science.

Multiple sensors can help identify errors

Suppose three nearby sensors report:

20.1°C

20.2°C

37.8°C

Perhaps the third sensor is faulty.

Redundant observations allow comparison.

This is another reason sensor networks often use multiple sources.

The system can ask:

Which observation is consistent with everything else we know?

Models can estimate hidden state

Recall the previous lesson.

The true state may not be directly known.

We have noisy observations.

So instead of assuming:

measurement = state

we estimate:

state given measurements.

Conceptually:

NOISY OBSERVATIONS

MODEL OF SYSTEM

↓

ESTIMATED STATE

This is the foundation of state estimation.

State estimation

Imagine a robot trying to estimate its location.

GPS says:

position A ± 5 metres

Wheel sensors say:

position B ± 1 metre

A camera suggests:

position C ± 0.5 metres

The robot combines these observations according to their reliability.

It does not simply ask:

Which sensor is correct?

It asks:

Given all the evidence and uncertainty, where am I most likely to be?

This is a much richer way to reason about measurement.

Measurements should come with uncertainty

Imagine two sensors.

Sensor A

temperature = 20.0°C ± 0.1°C

Sensor B

temperature = 20.0°C ± 5°C

Both report the same central value.

But they provide very different information.

A model that receives only:

20.0°C

cannot distinguish them.

Ideally, observations should include some representation of confidence or uncertainty.

Measurement uncertainty is information

Instead of storing only:

value = 20.0

we might store:

value = 20.0
uncertainty = ±0.2
sensor = A
timestamp = 14:02
calibration_date = 2026-07-01

Now the measurement carries information about:

what was observed,
how confidently,
by what instrument,
when.

This is richer data.

Provenance matters again

Recall the previous lesson on missing data.

We distinguished:

measured

from:

estimated.

Measurement error adds another layer.

A value may be:

directly measured,
measured by a low-quality sensor,
measured by a calibrated reference instrument,
inferred from neighbouring observations,
manually entered,
generated by another model.

These are different forms of evidence.

Knowing the provenance helps us interpret the number.

Uncertainty can propagate

Suppose a temperature measurement is uncertain.

That temperature feeds a demand forecast.

The demand forecast feeds an optimisation model.

The optimisation determines how much capacity to schedule.

The chain becomes:

UNCERTAIN MEASUREMENT

↓

UNCERTAIN STATE

↓

UNCERTAIN PREDICTION

↓

DECISION

The uncertainty did not disappear simply because the number entered a model.

A well-designed system should propagate relevant uncertainty through the chain.

False precision can propagate too

Suppose a sensor reports:

20°C ± 2°C

A model converts this into:

predicted demand = 4,283.712 MW

The output looks extraordinarily precise.

But the input uncertainty has not vanished.

The decimal places may create an illusion of certainty.

AI systems can produce highly precise outputs from highly uncertain inputs.

We should always distinguish:

computational precision

from:

epistemic certainty.

Measurement error and classification

Measurement error is not limited to numerical values.

Suppose a medical test predicts:

positive

or:

negative.

The test may sometimes be wrong.

There are two important possibilities:

false positive

and:

false negative.

These errors can have very different consequences.

We will examine this in detail when we study how predictions should be evaluated.

Sensors can classify too

A smoke detector effectively makes a classification:

smoke detected

or:

smoke not detected.

It may produce:

false alarms,
missed fires.

The quality of the measurement system cannot therefore be described by a single notion of accuracy.

We need to consider the consequences of different errors.

Measurement quality depends on purpose

Suppose GPS has an error of:

±5 metres.

For:

Which city am I in?

that is excellent.

For:

Which traffic lane am I in?

it may be inadequate.

Likewise, a thermometer accurate to:

±1°C

may be perfectly adequate for home heating.

It may be inadequate for a sensitive laboratory process.

There is no universally perfect measurement.

Measurement quality should be judged relative to the decision it supports.

We do not need perfect measurements

This mirrors the previous lesson on missing data.

We do not need:

perfect observations of everything

to build useful systems.

We need measurements that are:

good enough for the decision

together with:

an understanding of their uncertainty.

A navigation system can operate with imperfect GPS.

A weather forecast can use noisy measurements.

A robot can combine imperfect sensors.

An electricity system can estimate conditions from incomplete telemetry.

Robust systems are designed around imperfect information.

Sometimes more measurement is valuable

Suppose uncertainty is too high to make a safe decision.

The system may request another observation.

A robot might:

move its camera,
activate another sensor,
slow down,
wait for GPS.

A doctor might:

order another test.

An electricity operator might:

request more telemetry.

So the system can ask:

Would another measurement materially improve the decision?

This is the value-of-information idea again.

Sometimes another measurement adds almost nothing

Suppose we already have:

100 highly accurate sensors

measuring the same stable quantity.

Adding sensor 101 may provide almost no useful information.

Measurement has costs.

So the goal is not:

maximum measurement

but:

sufficient information for good decisions.

Error and uncertainty are not the same thing

It is useful to distinguish these concepts.

Error is the difference between a measurement and the true value.

Uncertainty describes how unsure we are about what that error might be.

If the true value is known, we can calculate the error.

But often the true value is precisely what we do not know.

So in real systems we frequently work with uncertainty rather than known error.

For example:

measurement = 20°C

uncertainty = ±0.5°C

We do not know the exact error.

We know something about its plausible range.

Aleatoric and epistemic uncertainty

Later we will explore different kinds of uncertainty more carefully.

But one useful distinction can be introduced here.

Aleatoric uncertainty

Uncertainty arising from inherent randomness or noise.

For example:

sensor noise,
unpredictable variation.

Epistemic uncertainty

Uncertainty arising because we do not know enough.

For example:

poor calibration,
limited data,
uncertain model parameters.

The distinction matters because additional information may reduce some forms of uncertainty more easily than others.

Measurement error can create apparent patterns

Suppose two sensors are systematically calibrated differently.

Sensor A is installed in northern locations.

Sensor B is installed in southern locations.

If Sensor B consistently reads:

1°C higher

we might incorrectly conclude that the south is warmer by 1°C.

The apparent geographic pattern is partly a measurement artefact.

This is why measurement systems must be considered when interpreting patterns in data.

Measurement changes can look like real change

Suppose a hospital changes how it records a diagnosis.

Recorded cases suddenly increase.

Did the disease become more common?

Perhaps.

Or perhaps the measurement and classification system changed.

Historical datasets often contain changes in:

definitions,
instruments,
reporting rules,
software systems.

A discontinuity in the data may therefore reflect a change in measurement rather than a change in reality.

Better measurement can make a problem appear worse

Suppose a city installs much better air-quality sensors.

Recorded pollution increases.

It might look as though pollution became worse.

But perhaps the previous measurement system simply failed to detect certain pollutants accurately.

Better observation can increase measured incidence without increasing the underlying phenomenon.

This is common in many domains.

AI can inherit historical measurement systems

Suppose an AI model is trained on 20 years of historical records.

Those records were produced by:

particular sensors,
institutional definitions,
human judgements,
administrative systems.

The model learns from the measured world.

Not directly from the world itself.

If historical measurement systems contained biases, the model may inherit them.

This becomes particularly important when we later examine fairness.

A model can amplify measurement problems

Suppose a biased measurement is used only occasionally by a human.

Its effect may be limited.

Now train an automated model using millions of those measurements.

Deploy the model to make millions of decisions.

A small systematic measurement bias can potentially become a large systematic decision bias.

The chain is:

MEASUREMENT BIAS

↓

TRAINING DATA

↓

MODEL

↓

PREDICTIONS

↓

DECISIONS AT SCALE

Scale amplifies the importance of measurement design.

Ask how the number was created

Whenever you see a number in a dataset, ask:

What exactly is being measured?
What instrument or process produced it?
At what time?
At what location?
At what resolution?
How accurate is the instrument?
How precise is it?
Is the error random or systematic?
Has the instrument been calibrated?
Can it saturate?
Does it have a detection limit?
Could the environment affect the measurement?
Is the value directly observed or inferred?
Has the measurement process changed over time?
Is the variable actually a proxy for something else?
How much uncertainty should accompany the value?
Does the measurement quality matter for the decision being made?

These questions turn a number from an apparent fact into what it really is:

evidence produced by a measurement process.

The central idea

Data does not arrive directly from reality.

It passes through measurement.

REALITY

↓

MEASUREMENT SYSTEM

↓

OBSERVATION

↓

DATA

↓

MODEL

Measurement systems have:

resolution,
noise,
limits,
biases,
uncertainty.

That is not necessarily a problem.

It is a property of observing the physical and social world.

Useful intelligence does not require perfect measurements. It requires measurements whose uncertainty is understood well enough for the decisions being made.

The important distinction is therefore not:

perfect data versus bad data

but:

what do we know, how did we measure it, and how uncertain are we?

This strengthens the framework we developed in the previous lesson:

PARTIAL + IMPERFECT OBSERVATIONS

↓

INFERENCE

↓

ESTIMATED STATE + UNCERTAINTY

↓

PREDICTION

↓

DECISION

We now have another reason to be cautious when interpreting patterns in datasets.

Two variables may appear to move together.

But that does not automatically tell us why.

And even when a relationship is real, it does not necessarily mean that changing one variable will change the other.

That brings us to one of the most important distinctions in data analysis:

correlation and causation.