Variables, features and labels

Data becomes useful when we organise it.

A dataset may contain thousands, millions or billions of observations, but a model needs some structure in order to learn from them.

That structure often comes from three closely related ideas:

variables
features
labels

These terms are sometimes used interchangeably, but they describe different roles that information can play.

Understanding the distinction is important because machine learning is fundamentally about relationships between variables.

Variables

A variable is a quantity or characteristic that can take different values.

Examples include:

age,
temperature,
income,
electricity demand,
speed,
location,
number of purchases,
blood pressure.

Suppose we have a dataset describing houses:

House	Bedrooms	Floor area	Distance to city centre	Sale price
A	2	85 m²	4 km	€420,000
B	3	120 m²	8 km	€510,000
C	4	160 m²	12 km	€590,000

The columns:

Bedrooms
Floor area
Distance to city centre
Sale price

are variables.

Each row gives the values of those variables for one observation.

Observations and variables

A useful way to think about a dataset is:

ROWS → observations

COLUMNS → variables

For example:

Person	Age	Income	Commute time
A	24	€32,000	40 min
B	51	€68,000	25 min
C	37	€51,000	55 min

Each person is an observation.

Each column describes a variable.

This structure allows us to ask questions such as:

Does income tend to increase with age?
Does commute time vary by location?
Can one variable help predict another?

Much of statistical analysis begins with relationships between variables.

Variables can represent many kinds of things

Some variables are numerical.

For example:

temperature = 18.2°C
income = €45,000
distance = 7.3 km

Others are categorical.

For example:

colour = red
employment status = employed
transport mode = train
region = west

Some variables describe time.

For example:

date,
hour,
day of week.

Others describe space.

For example:

latitude,
longitude,
postcode,
network node.

The type of variable affects how we can analyse it.

Features

In machine learning, the variables used by the model as inputs are often called features.

Suppose we want to predict house prices.

Our model might receive:

number of bedrooms,
floor area,
distance to city centre,
building age.

These are the model's features.

Conceptually:

FEATURES → MODEL → PREDICTION

The features provide information the model can use to estimate an unknown outcome.

Features describe what the model gets to see

Imagine two models predicting whether a train will arrive late.

Model A receives only:

departure time.

Model B receives:

departure time,
weather,
route,
current delay,
congestion,
previous service status.

Model B has access to a richer set of features.

That does not guarantee that it will perform better, but it has more information from which to learn.

This gives us an important principle:

Features define the information available to the model.

If important information is absent from the features, the model cannot use it directly.

Feature selection

We do not always want to give a model every variable available.

Some variables may be:

irrelevant,
duplicated,
noisy,
expensive to collect,
privacy-sensitive,
misleading.

Choosing which variables to include is called feature selection.

Suppose we want to predict electricity demand.

Potential features might include:

hour of day,
temperature,
day of week,
previous demand,
electricity price,
holiday status.

Perhaps we also have:

colour of the control-room walls.

That variable is unlikely to contain useful information about demand.

Including more variables does not automatically improve a model.

Features can be created

Features do not always have to be raw measurements.

We can derive new features from existing variables.

Suppose we have a timestamp:

2026-08-06 18:30

From that, we might derive:

hour = 18
day of week = Thursday
month = August
weekend = false

These derived variables may make important patterns easier for a model to learn.

Similarly, from:

distance travelled
time taken

we might create:

average speed

Creating useful variables from existing data is called feature engineering.

Feature engineering

Feature engineering uses knowledge about the problem to construct representations that may help a model.

For example, a model predicting electricity demand might use:

current temperature,
temperature squared,
previous-hour demand,
average demand over the previous 24 hours,
whether today is a public holiday.

A financial model might create:

debt-to-income ratio,
average monthly spending,
recent payment frequency.

A transport model might create:

distance to nearest station,
average historical congestion,
time since previous service.

The original observations have not changed.

The representation has.

This connects directly to an earlier idea:

Models depend on how we choose to represent the world.

Labels

A label is the value we want a supervised machine-learning model to learn to predict.

Suppose our housing dataset looks like this:

Bedrooms	Floor area	Distance	Sale price
2	85 m²	4 km	€420,000
3	120 m²	8 km	€510,000
4	160 m²	12 km	€590,000

If our goal is to predict:

Sale price

then sale price is the label.

The remaining variables might be used as features.

So:

Bedrooms + Floor area + Distance → MODEL → Sale price

The model learns from historical examples in which the correct output is known.

Features and labels depend on the question

The same variable can play different roles in different problems.

Suppose a dataset contains:

age,
income,
occupation,
postcode.

If we want to predict income, then:

income = label

and the others may be features.

If we want to predict occupation, then:

occupation = label

and income may become a feature.

There is nothing inherently special about a variable that makes it a label.

Its role depends on the prediction task.

Inputs and outputs

Another way to describe this is:

FEATURES = INPUTS

LABEL = TARGET OUTPUT

Suppose we want to predict tomorrow's temperature.

Features might include:

today's temperature,
air pressure,
humidity,
wind direction,
recent weather.

The label during training might be:

temperature 24 hours later

The model sees many historical examples:

inputs → known outcome

and attempts to learn the relationship between them.

Later, when the future outcome is not yet known, it receives new inputs and generates a prediction.

Supervised learning

Machine learning that uses known labels during training is called supervised learning.

The model is shown examples such as:

FEATURES → LABEL

For example:

email text → spam

medical measurements → diagnosis

house characteristics → sale price

image pixels → cat

The label provides the answer the model is trying to learn.

During training, the model's prediction can be compared with the known label.

That comparison allows us to calculate an error.

And that error can be used to improve the model.

This will become central when we study loss functions and gradient descent.

Where do labels come from?

This is an extremely important question.

Labels may appear to be ground truth.

But labels themselves usually come from some process.

For example:

Spam detection

The label might come from:

a human reviewer,
user reports,
existing rules.

Medical diagnosis

The label might come from:

a clinician,
a laboratory test,
a later health outcome.

Hiring

The label might be:

hired,
not hired.

But that reflects a previous hiring decision.

Credit

The label might be:

repaid,
defaulted.

But perhaps some people were never offered credit and therefore never generated a repayment outcome.

Labels have origins.

Those origins matter.

Labels can contain human judgement

Suppose we train a model to identify whether online comments are:

toxic

or:

not toxic

Human reviewers may assign the labels.

But different reviewers may disagree.

One person may interpret sarcasm differently.

Another may consider particular language offensive.

Cultural context may matter.

The resulting label is therefore not always an objective property of the comment.

It may reflect human judgement.

This does not make supervised learning impossible.

It simply means we should avoid assuming:

label = unquestionable truth

Historical decisions can become labels

Consider a model trained to predict whether a job applicant is:

successful

Suppose the historical label is:

was hired

But being hired is not the same as being capable of doing the job.

The label reflects the organisation's previous hiring decisions.

If those decisions contained bias, the label may encode that bias.

The model may then learn:

Which applicants resembled people historically selected by the organisation?

rather than:

Which applicants would actually perform best?

This distinction can be subtle but enormously consequential.

Observed labels and constructed labels

Some labels are relatively direct observations.

For example:

Did the machine fail within 24 hours?

That outcome can often be observed.

Others are constructed.

For example:

high-risk customer

Someone has to define what counts as "high risk".

Perhaps:

risk score > 70

But where did the risk score come from?

Labels can therefore sit several layers away from direct observation.

For example:

REALITY → MEASUREMENT → SCORE → CATEGORY → LABEL

Each step introduces choices.

Labels can arrive later

Sometimes the correct outcome is not known immediately.

Suppose a bank wants to predict whether a loan will default.

When the loan is issued, the label does not yet exist.

The outcome may only become known months or years later.

This creates an important temporal relationship:

FEATURES NOW → LABEL LATER

Many predictive systems have exactly this structure.

The model learns relationships between earlier conditions and later outcomes.

This connects directly to our discussion of prediction as a bridge through time.

Some labels are easier to obtain than others

Suppose we want to train an autonomous vehicle.

Images are relatively easy to collect.

But the model may need labels identifying:

cars,
pedestrians,
bicycles,
traffic lights,
road signs.

Historically, humans often had to annotate those images.

That process can be expensive and slow.

Modern AI increasingly uses techniques that reduce dependence on manually labelled datasets.

This is one of the reasons self-supervised learning and foundation models became so important.

We will explore that later.

Unlabelled data

Not all datasets contain labels.

Suppose we have millions of customer transactions but no predefined target.

We might want to discover:

groups of similar customers,
unusual transactions,
underlying patterns.

This is different from supervised learning.

The model is not given a correct answer for every example.

Instead, it attempts to identify structure in the data.

This family of approaches is often called unsupervised learning.

The distinction is useful:

supervised learning → learn from features and labels

unsupervised learning → find structure without explicit labels

Self-supervised learning

Modern AI often uses an interesting middle ground.

The dataset may not contain human-created labels, but the system creates learning targets from the data itself.

Consider language.

Given:

The cat sat on the ___

the original text already contains the next word.

We can hide part of the sequence and ask the model to predict it.

The data provides its own training target.

This is one form of self-supervised learning.

Large language models are trained extensively using this principle.

Massive quantities of text can therefore become training examples without humans manually labelling every sentence.

A label can be a number

Labels do not have to be categories.

Suppose we predict:

house price = €420,000

The target is numerical.

This type of prediction is called regression.

Examples include predicting:

temperature,
electricity demand,
price,
journey time,
energy consumption.

The model attempts to estimate a continuous or numerical value.

A label can be a category

Suppose we predict:

spam

or:

not spam

This is a classification problem.

Other examples include:

cat / dog,
fraud / not fraud,
disease / no disease,
low / medium / high risk.

The target belongs to one of a set of categories.

We will explore regression and classification in more detail in the machine-learning module.

Labels can have multiple classes

Classification does not have to involve only two outcomes.

A model might classify an image as:

cat,
dog,
horse,
bird.

Or classify weather as:

sunny,
cloudy,
rainy,
snowy.

This is multiclass classification.

The label identifies which category the example belongs to.

An observation can have many labels

Sometimes one observation belongs to several categories at once.

An image might contain:

a person,
a bicycle,
a traffic light,
a car.

This is different from choosing exactly one category.

Similarly, a song might be labelled:

electronic,
instrumental,
upbeat.

The way labels are structured depends on what the task requires.

Again, representation matters.

Features can contain the answer accidentally

There is an important modelling mistake called data leakage.

Suppose we want to predict whether a patient will be admitted to hospital.

Our dataset contains:

symptoms,
age,
blood pressure,
hospital bed number.

But perhaps a bed number is only assigned after someone has already been admitted.

The feature effectively reveals the future outcome.

A model using it may appear extraordinarily accurate.

But it has learned from information that would not actually be available when the real prediction needs to be made.

This gives us a critical principle:

Features must reflect information genuinely available at the time of prediction.

Otherwise, model performance can be misleading.

Time matters when creating features

Suppose at 12:00 we want to predict electricity demand at 13:00.

Valid features might include:

demand at 11:30,
demand at 12:00,
current temperature,
weather forecast.

But we cannot include:

demand at 12:30

if that information was not yet available at the prediction time.

The dataset used for training must respect the boundary between:

information available now

and:

information that belongs to the future.

Otherwise the model is effectively being allowed to look ahead in time.

Features can encode space

Location can also be used as a feature.

For example:

latitude,
longitude,
postcode,
region,
road segment,
network node.

Spatial features can be extremely informative.

But as we saw earlier, they may also encode other characteristics indirectly.

Postcode may correlate with:

income,
property values,
ethnicity,
access to services.

So a seemingly innocent location feature can become a proxy for sensitive information.

This will matter later when we examine fairness.

Features can encode history

Suppose we predict whether someone will repay a loan.

Features might include:

previous repayments,
previous defaults,
account history.

These variables describe the person's past.

That history may be highly predictive.

But it may also reflect previous access to opportunities.

Someone who was historically denied credit may have less repayment history simply because they had fewer opportunities to borrow.

Features are therefore not always neutral descriptions.

They can reflect interactions between people and institutions.

Feature importance does not automatically mean cause

Suppose a model finds that postcode is highly predictive of house price.

Does postcode cause the house price?

Not in a simple sense.

Postcode may be associated with:

location,
schools,
transport,
amenities,
local demand,
historical development.

A feature can be predictive without being causal.

This distinction will become important when we study correlation and causation.

Good features can make simple models powerful

Sometimes choosing the right representation matters more than choosing a complicated model.

Suppose we want to predict whether it is daytime.

We could train an enormous neural network using many raw observations.

Or perhaps we could use:

time,
latitude,
date.

A relatively simple model may then perform extremely well.

This illustrates an important idea in machine learning:

The representation of the problem can be as important as the algorithm used to solve it.

Modern deep learning often learns useful features automatically, but the underlying principle remains.

Deep learning changes feature engineering

Traditional machine learning often relied heavily on humans designing useful features.

For image recognition, engineers might manually calculate:

edges,
shapes,
textures.

For speech recognition:

frequency-based audio features.

For language:

word counts,
grammatical indicators.

Deep neural networks changed this significantly.

Instead of relying entirely on manually designed features, models can learn useful internal representations directly from rawer inputs.

For example:

pixels → learned representations → object prediction

or:

audio waveform → learned representations → speech recognition

or:

tokens → embeddings → learned representations → language prediction

This is one of the major shifts behind modern AI.

Inputs can become enormous

For simple datasets, a feature might be one number.

For modern AI, inputs can be much larger.

An image may contain millions of pixel values.

A piece of text may contain thousands of tokens.

An audio recording may contain millions of samples.

These raw inputs can be transformed into internal representations that act like learned features.

The terminology changes slightly, but the basic idea remains:

The model receives some representation of the world and uses it to infer something else.

Features and labels define the learning problem

Suppose we have a dataset.

Until we specify:

What information is the model allowed to use?

and:

What outcome are we asking it to predict?

we have not fully defined the machine-learning problem.

Features answer the first question.

Labels answer the second.

So:

FEATURES → MODEL → LABEL

is one of the most important structures in supervised machine learning.

Choosing the label chooses the task

This leads to a deeper point.

Suppose a social-media platform has many possible outcomes it could predict:

whether someone clicks,
whether someone shares,
whether someone learns something,
whether someone feels satisfied,
whether someone returns tomorrow,
whether something improves their wellbeing.

Which one should become the label?

That is not merely a technical question.

It determines what the model is trained to care about.

If we choose:

click

then the system learns to predict clicks.

If we optimise decisions using that prediction, the wider system may become organised around generating clicks.

So:

Choosing the target variable is partly choosing the behaviour we want the system to learn.

This becomes especially important when prediction is connected to optimisation.

Labels shape future systems

Consider a healthcare model.

We might train it to predict:

healthcare cost

But perhaps what we actually care about is:

healthcare need

Those are not necessarily the same.

A person who historically received little healthcare may have low cost despite having significant unmet need.

If cost becomes the label, the model learns the patterns associated with historical spending.

It does not automatically learn underlying need.

This is another example of why:

The measurable target may not perfectly represent the real objective.

From variables to decisions

Variables may appear like a purely technical topic.

But they sit near the beginning of a much larger chain:

WORLD

↓

MEASUREMENT

↓

VARIABLES

↓

FEATURES + LABELS

↓

MODEL

↓

PREDICTION

↓

DECISION

↓

ACTION

The way reality is translated into variables influences everything downstream.

If the variables poorly represent the world, the model inherits that limitation.

If the label poorly represents the objective, the model can become extremely good at solving the wrong problem.

Ask what each variable means

Whenever you encounter a machine-learning dataset, ask:

What does each variable represent?
How was it measured?
Is it directly observed or derived?
Which variables are being used as features?
What is the label?
Where did that label come from?
Would the features actually be available at prediction time?
Are important variables missing?
Could any feature act as a proxy for something sensitive?
Does the label represent what we really care about?

These questions help reveal what problem the model is actually learning.

The central idea

Variables give structure to data.

Features are the variables a model uses to make predictions.

Labels describe the outcomes a supervised model is asked to learn.

That sounds simple.

But choosing them defines the informational world of the model.

Features determine what the model gets to see. Labels determine what the model is taught to predict.

And neither choice is neutral.

The features reflect decisions about measurement and representation.

The labels reflect decisions about what counts as the outcome of interest.

Before a machine can learn anything useful, someone has already made these choices.

In the next lesson, we will look more closely at the different kinds of values variables can take, beginning with one of the most basic distinctions in data:

continuous and discrete data.