Measurement and observation

Before there can be data, something has to be observed.

Before a model can learn that the temperature is 18°C, a sensor has to interact with the physical world and produce a measurement.

Before an AI system can recognise a pedestrian, light has to reach a camera.

Before a hospital can analyse blood pressure, someone has to measure it.

Before a company can analyse customer behaviour, some part of that behaviour has to be recorded.

This creates a fundamental chain:

WORLD → OBSERVATION → MEASUREMENT → DATA

It is easy to overlook these first steps and begin with a spreadsheet, database or training dataset as though the data simply appeared.

It did not.

Every dataset has a measurement system behind it.

Understanding that system is essential because what we observe, how we measure it and what we fail to measure determine what the data can tell us.

Observation comes before data

Imagine someone walks into a shop.

Many things happen.

They:

enter at a particular time,
walk through particular aisles,
look at particular products,
pick some products up,
put some back,
speak to another person,
buy something,
leave.

The event itself is extraordinarily rich.

But perhaps the shop records only:

14:32 — Product A — €4.50 — purchased

That transaction becomes data.

The customer's hesitation before purchasing does not.

The product they considered but returned to the shelf does not.

Their reason for choosing Product A does not.

Unless some system observes and records those things, they disappear from the dataset.

This gives us an important distinction:

what happened

is not necessarily the same as:

what was observed

and that is not necessarily the same as:

what was recorded as data.

Observation is selective

No observer captures everything.

A camera records some wavelengths of light within its field of view.

A microphone records variations in air pressure within particular frequencies.

A thermometer measures temperature at a particular location.

A survey records answers to the questions that were asked.

A website records interactions that its software was designed to capture.

Every observation system therefore creates a window onto reality.

Inside the window, some things become visible.

Outside it, they do not.

Observation is always selective.

This is true whether the observer is a person, a sensor or a computer.

Measurement turns observations into representations

Measurement gives observations a defined representation.

Suppose we want to know how warm a room is.

We could say:

It feels warm.

That is an observation.

A thermometer might instead report:

23.4°C

Now the observation has been represented using:

a numerical scale,
a unit,
an instrument,
a measurement procedure.

This makes the observation easier to:

store,
compare,
transmit,
analyse,
use computationally.

Measurement therefore allows aspects of the world to become machine-readable information.

What exactly are we measuring?

This question is often harder than it first appears.

Some quantities are relatively straightforward.

We might measure:

length,
mass,
temperature,
voltage,
time.

But many things we care about are not directly measurable.

Consider:

health

There is no instrument that simply reports:

Health = 73.2

Instead, we measure things associated with health:

blood pressure,
heart rate,
blood markers,
symptoms,
diagnoses,
mobility,
medical history.

Or consider:

educational ability

We might measure:

examination results,
coursework,
attendance,
teacher assessments.

Or:

creditworthiness

We might use:

repayment history,
income,
debt,
missed payments.

In each case, the thing we care about is more complicated than the measurements used to represent it.

Proxies

When something cannot be measured directly, we often measure something else that we believe is related to it.

This is called a proxy.

For example:

examination score → proxy for educational attainment

income → possible proxy for ability to repay

clicks → proxy for interest

time spent viewing content → proxy for engagement

steps per day → proxy for physical activity

Proxies are enormously useful.

But they introduce assumptions.

Does clicking something mean that you liked it?

Does spending a long time reading something mean you found it valuable?

Does examination performance capture everything we mean by educational ability?

Not necessarily.

This gives us an important principle:

What is easy to measure may not be the same as what actually matters.

Measurement requires a definition

Suppose we want to measure unemployment.

Who counts as unemployed?

Someone without a job who is actively looking for work?

Someone who wants work but has stopped searching?

Someone working one hour per week?

Someone caring for children but willing to work under different circumstances?

Before we can produce an unemployment statistic, we need a definition.

That definition determines who is counted.

Change the definition and the measured value may change even if nobody's real circumstances have changed.

Measurement therefore often begins with a conceptual decision:

What exactly do we mean by the thing we are trying to measure?

Operational definitions

Turning an abstract concept into something measurable is sometimes called operationalisation.

Suppose we want to measure:

customer satisfaction

We might operationalise it as:

the answer to the question "How satisfied are you with the service?" on a scale from 1 to 5.

Now we can collect data.

But notice what has happened.

The complex concept:

customer satisfaction

has become:

a number between 1 and 5

That may be useful.

But the number is a representation of satisfaction, not satisfaction itself.

The measurement process has created an abstraction.

Units matter

Measurements require units.

Consider the value:

It could represent:

50 metres,
50 kilometres,
50 kilograms,
50 seconds,
50°C,
€50.

Without units, numerical data can be meaningless.

Units also affect scale.

A distance of:

1 kilometre

can also be represented as:

1,000 metres

The physical distance has not changed.

The numerical representation has.

AI systems generally receive numerical values, not physical quantities themselves.

That means the representation and units used in a dataset can affect how models process information.

Precision is not the same as accuracy

Suppose a thermometer displays:

21.384726°C

That looks extremely precise.

But what if the thermometer is poorly calibrated and the true temperature is closer to:

23°C?

The measurement is precise in the sense that it contains many decimal places.

It is not accurate.

This distinction matters.

Precision

How finely or consistently something is measured.

Accuracy

How close the measurement is to the quantity we actually want to know.

A measurement can be:

precise and accurate,
precise but inaccurate,
imprecise but approximately accurate,
both imprecise and inaccurate.

More decimal places do not automatically mean better information.

Resolution matters

Sensors also have limited resolution.

Suppose a temperature sensor records only whole degrees.

A true temperature of:

20.1°C

might be recorded as:

20°C

A true temperature of:

20.4°C

might also be recorded as:

20°C

The sensor cannot distinguish between them.

Information has been lost.

The same issue appears in:

images,
sound recordings,
location systems,
medical instruments,
financial data.

Every measurement system has limits to what it can distinguish.

Sampling in time

Many systems change continuously.

But we usually measure them at discrete moments.

Suppose the actual temperature changes continuously:

... → 20.1 → 20.2 → 20.4 → 20.7 → 21.0 → ...

Perhaps our sensor records only once every hour.

We receive:

Time	Temperature
09:00	20°C
10:00	21°C
11:00	23°C

What happened between those observations?

We do not know directly.

The system continued evolving.

Our measurement system simply did not record it.

This is sampling.

We observe a changing system at selected points.

Sampling frequency changes what we can see

Imagine measuring electricity demand once per day.

We might observe:

Day	Average demand
Monday	30 GW
Tuesday	31 GW
Wednesday	30 GW

The system appears relatively stable.

Now measure every hour.

Suddenly we see:

morning peaks,
evening peaks,
overnight lows.

Measure every second and we may observe even faster fluctuations.

The physical system is the same.

The measurement resolution has changed.

How often we observe a system determines which patterns become visible.

Spatial sampling matters too

Suppose we want to measure air pollution across a city.

If we install one sensor, we obtain one view of the city.

If that sensor happens to be in a park, we might conclude that air quality is relatively good.

Install sensors beside:

major roads,
schools,
industrial areas,
residential neighbourhoods,

and a much more complicated picture appears.

Again, the city has not changed.

The measurement system has.

Where we observe determines what we see.

Sensors have locations

Consider the statement:

Temperature = 18°C

Where was it measured?

Inside?

Outside?

In sunlight?

In shade?

At ground level?

On top of a building?

Near an air-conditioning outlet?

Location can radically affect a measurement.

This is why good datasets often need metadata describing:

where measurements were taken,
when they were taken,
how they were taken.

A value without its measurement context can be misleading.

Instruments affect observations

Measurement instruments are not perfect windows onto reality.

They have characteristics of their own.

A camera has:

resolution,
field of view,
exposure,
sensitivity,
lens distortion.

A microphone has:

frequency response,
directionality,
sensitivity.

A temperature sensor has:

accuracy,
response time,
calibration.

A GPS receiver has:

positional uncertainty,
update frequency.

The data produced therefore depends partly on the instrument.

Two different instruments observing the same event may produce different data.

Calibration

Measurement instruments often need to be calibrated.

Suppose a scale consistently reports:

actual mass + 2 kg

A person who actually weighs 70 kg will appear to weigh 72 kg.

The instrument is systematically biased.

Calibration attempts to ensure that the measurement produced by an instrument corresponds appropriately to a known reference.

Without calibration, systematic errors can enter datasets.

If millions of measurements are collected from a miscalibrated sensor, having more data does not solve the problem.

We simply obtain more incorrect measurements.

Random error

Some measurement errors vary unpredictably.

Suppose we repeatedly measure the same quantity and obtain:

20.1

19.9

20.2

20.0

19.8

Small fluctuations may arise from:

sensor noise,
environmental variation,
rounding,
electronic noise.

This is different from a sensor that always reports a value 2 units too high.

One produces variable noise.

The other produces systematic bias.

Both affect data differently.

We will examine measurement error in more detail later in this module.

Human observation has measurement problems too

Measurement error is not limited to machines.

Humans produce data as well.

Consider a doctor recording a patient's pain on a scale from 1 to 10.

Two patients experiencing similar pain might choose different numbers.

The same patient might give different answers on different days.

Two doctors might interpret symptoms differently.

Human-generated data can be influenced by:

judgement,
memory,
expectations,
language,
incentives,
social pressure.

This does not make the data useless.

It means we need to understand how it was produced.

Observation can change behaviour

Sometimes the act of measuring a system changes the system itself.

Suppose workers know their productivity is being monitored.

They may behave differently.

Suppose students know a particular metric will determine school rankings.

Schools may begin optimising specifically for that metric.

Suppose social-media creators know engagement is being measured.

They may adapt their content to maximise engagement.

The measurement is no longer passive.

It becomes part of the system.

This creates another feedback loop:

MEASURE → RESPOND → CHANGE BEHAVIOUR → NEW MEASUREMENT

We will encounter this idea repeatedly when we study intelligent services and reflexive systems.

When a measure becomes a target

Suppose a company wants to improve customer service.

It chooses:

average call duration

as a performance measure.

Shorter calls are interpreted as greater efficiency.

Employees are therefore encouraged to reduce call duration.

What might happen?

Calls may become shorter.

But customer problems may not actually be solved.

The measurement improved.

The underlying objective may not have.

This illustrates an important problem:

When a measure becomes a target, people and systems may optimise the measure rather than the thing it was intended to represent.

This becomes extremely important when AI systems optimise measurable objectives.

The machine only sees what was measured

Imagine an AI system deciding whether someone should receive a loan.

The model receives:

income,
debt,
employment status,
repayment history.

It does not directly observe:

the person's intentions,
their future circumstances,
their complete financial resilience,
unexpected events that may occur next year.

The model sees a measured representation.

It then learns relationships within that representation.

Conceptually:

PERSON

↓

MEASUREMENT SYSTEM

↓

VARIABLES

↓

AI MODEL

The AI does not know the person.

It knows the data produced about the person.

That distinction matters.

Observation is not state

This connects to an idea introduced earlier in the course.

The state of a system represents what we believe matters about its current condition.

But we may not be able to observe that state directly.

Consider an autonomous vehicle.

Its true environment includes:

the exact position of every vehicle,
their velocities,
pedestrians,
road conditions,
intentions of other drivers.

The vehicle receives observations from:

cameras,
radar,
lidar,
GPS,
other sensors.

Those observations are incomplete and noisy.

The machine must infer its best estimate of the state of the world.

So:

WORLD → OBSERVATIONS → ESTIMATED STATE

The observation is not the state.

It is evidence about the state.

Hidden states

Many important things cannot be directly observed.

Suppose an industrial machine is gradually degrading internally.

We cannot directly see:

health = 63%

Instead, we observe:

vibration,
temperature,
sound,
electrical consumption.

From those signals we infer something about the machine's hidden condition.

Likewise, we cannot directly observe:

someone's future credit risk,
a customer's underlying preference,
the exact state of the atmosphere,
another driver's intention.

We observe signals and infer hidden states.

This idea connects measurement directly to probability, prediction and AI.

Multiple measurements can reveal more

A single observation may be ambiguous.

Suppose a robot's camera suggests an object is ten metres away.

Another sensor estimates twelve metres.

A third estimates eleven metres.

Combining several measurements may give a better estimate than relying on one alone.

This process is sometimes called sensor fusion.

Humans do something similar.

We combine:

sight,
sound,
touch,
memory,
context

to form an understanding of what is happening.

Intelligent machines increasingly combine multiple sources of information in the same way.

Different sensors see different worlds

Imagine an autonomous vehicle.

A camera sees:

colours,
markings,
signs,
visual objects.

Radar can help detect:

distance,
velocity.

Lidar can provide information about:

three-dimensional structure,
distance.

GPS provides information about:

geographic position.

No single sensor perfectly describes the environment.

Each produces a different representation.

Together, they provide a richer basis for estimating the state of the world.

This is one reason multimodal AI can be so powerful.

Different forms of data reveal different aspects of reality.

Measurement systems can create bias

Suppose a medical device performs less accurately for some groups of patients than others.

Even if the AI model itself is mathematically identical for everyone, its input data may have different quality across groups.

Or suppose facial-recognition training images are captured primarily under particular lighting conditions.

Performance may deteriorate in environments that were poorly represented.

Bias can therefore enter an AI system before machine learning begins.

The problem may originate in:

the sensor,
the measurement procedure,
the sampling process,
the categories,
the labels.

This is why fairness cannot be understood solely by examining an algorithm.

Absence of measurement is information too

Sometimes what is not measured tells us something about the system that produced the dataset.

Imagine a healthcare dataset containing far fewer records from a particular community.

Does that mean people in that community are healthier?

Perhaps.

But perhaps they have:

poorer access to healthcare,
less insurance coverage,
longer travel distances,
lower trust in institutions.

The absence of observations may reflect absence of access rather than absence of need.

This is an important warning:

No recorded event does not necessarily mean no event occurred.

Sometimes it means nobody observed it.

Measurement creates visibility

What societies measure becomes visible.

What is visible can be:

analysed,
compared,
managed,
optimised.

What is not measured can become invisible to decision-making systems.

If a transport system measures vehicle journey times but not pedestrian safety, optimisation may focus on vehicles.

If a company measures productivity but not employee wellbeing, its systems may optimise productivity.

If an online platform measures engagement but not whether users found something genuinely valuable, engagement becomes the visible objective.

Measurement therefore has consequences.

Choosing what to measure is partly choosing what the system is capable of caring about.

AI makes measurement choices more consequential

A human analyst might examine a dataset occasionally.

An AI system can use measurements to make millions of decisions automatically.

This increases the consequences of measurement design.

Suppose a small measurement bias enters an automated system.

If the system makes:

one decision

the effect may be small.

If it makes:

one billion decisions

the same systematic bias can become enormously consequential.

Automation scales both good measurement and bad measurement.

The complete measurement chain

We can now describe the process more carefully.

Start with:

REALITY

Something happens.

Then:

OBSERVATION

Some part of the event becomes detectable.

Then:

MEASUREMENT

The observation is represented using an instrument, category, scale or procedure.

Then:

RECORDING

The measurement is stored.

Then:

DATA

The recorded observations become available for analysis.

So:

REALITY → OBSERVATION → MEASUREMENT → RECORDING → DATA

And only then do we reach:

DATA → MODEL → PREDICTION

This means there is an entire system of decisions and possible errors before machine learning even begins.

Measurement sits inside the wider feedback loop

Recall the framework from Module 1:

PAST → DATA → MODEL → PREDICTION → DECISION → ACTION → FUTURE

We can now expand the first step:

WORLD

↓

OBSERVATION

↓

MEASUREMENT

↓

DATA

↓

MODEL

↓

PREDICTION

↓

DECISION

↓

ACTION

↓

NEW WORLD

And then the new world is observed again.

This creates a loop:

WORLD → MEASURE → MODEL → PREDICT → DECIDE → ACT → WORLD

What we measure influences the model.

The model influences decisions.

Decisions influence the world.

And the changed world determines what gets measured next.

Ask about the measurement system

Whenever you encounter data, ask:

What real-world thing is this supposed to represent?
Was it observed directly or inferred?
How was it measured?
What instrument or process produced it?
What units were used?
How precise is the measurement?
How accurate is it?
When was it measured?
Where was it measured?
How frequently was it measured?
What could the measurement system not observe?
Are we measuring the thing we care about or a proxy for it?
Could the act of measurement change behaviour?

These questions help us understand what the dataset actually means.

The central idea

Measurement is the bridge between the world and data.

It converts aspects of reality into representations that can be stored and processed.

But every measurement system makes choices.

It decides:

what becomes visible

what remains invisible

how reality is represented

at what resolution

at what time

at what location

with what uncertainty

Those choices become embedded in the data.

And because artificial intelligence learns from data, they can eventually become embedded in predictions and decisions too.

An AI system can only learn from the world that its measurement system allows it to see.

Understanding data therefore begins not with algorithms, but with observation.

Before asking what a model has learned, we should first ask:

What did we actually measure?