Enleashed

Every prediction problem begins with two basic questions:

What information does the model receive?

and:

What do we want it to produce?

The information going into the model is called the input.

The thing we want the model to predict is called the output.

At its simplest:

INPUTS → MODEL → OUTPUT

For example:

house size + location + age

↓

model

↓

predicted house price

Or:

email text

↓

model

↓

spam / not spam

The entire machine-learning problem is built around defining this relationship clearly.

Inputs are the information available to the model

Suppose we want to predict tomorrow's electricity demand.

Possible inputs might include:

recent electricity demand,
temperature,
time of day,
day of week,
season,
public holidays.

These are pieces of information that may help predict the target.

We can represent them conceptually as:

X = inputs

The model then uses X to estimate the output.

Outputs are the quantities we want to know

The output is often represented as:

For example:

Y = electricity demand tomorrow at 18:00

The model tries to learn a relationship:

X → Y

During prediction, it produces:

where Ŷ means:

our estimate of Y.

So:

X → MODEL → Ŷ

Later, when reality is observed, we can compare:

with:

A simple numerical example

Suppose the inputs are:

temperature = 5°C

time = 18:00

previous demand = 38 GW

The model predicts:

Ŷ = 41 GW

Tomorrow at 18:00, the actual demand is:

Y = 42 GW

The inputs were the information available beforehand.

The output was the prediction.

The target was the value we were trying to estimate.

One input or many inputs

Some prediction problems use only one input.

For example:

temperature → ice cream sales

But most real machine-learning problems use many.

For example:

house price might depend on:

floor area,
location,
number of bedrooms,
building age,
condition,
local amenities.

Together, these inputs form an input vector.

Conceptually:

x = [x₁, x₂, x₃, ..., xₙ]

Each component represents one piece of information.

Inputs are often called features

In machine learning, input variables are commonly called features.

For a house-price model:

Feature 1 = floor area

Feature 2 = distance from city centre

Feature 3 = age

Feature 4 = number of bedrooms

The complete feature vector describes the example presented to the model.

Later we will look at feature engineering in more detail.

Outputs are often called targets or labels

The terminology depends on the type of problem.

For numerical prediction:

target

is common.

For classification:

label

is common.

For example:

Regression

Input: house characteristics

Target: sale price

Classification

Input: email text

Label: spam / not spam

The basic structure is the same.

Inputs and outputs define the learning problem

Suppose we have a dataset containing:

Temperature	Time	Demand
5°C	18:00	42 GW
10°C	18:00	39 GW
15°C	18:00	36 GW

If the task is:

Predict demand from temperature and time

then:

Inputs = Temperature + Time

Output = Demand

But we could use the same dataset for a different task.

For example:

Predict temperature from demand and time.

Now:

Inputs = Demand + Time

Output = Temperature

The data is the same.

The prediction problem changed.

The direction matters

Machine learning is not simply:

Find relationships in the data.

A predictive task defines a direction:

available information

towards:

unknown target.

That direction reflects what will actually be known at prediction time.

This is important.

A variable can be highly correlated with the target and still be unusable if it will not be available when the prediction is needed.

Inputs must exist at prediction time

Suppose we want to predict:

whether a patient will need intensive care tomorrow.

A model cannot legitimately use:

whether the patient was transferred to intensive care tomorrow

as an input.

That information belongs to the future.

It was not available when the prediction had to be made.

This would be data leakage.

The information boundary

A useful way to think about the problem is:

KNOWN AT TIME OF PREDICTION

↓

INPUTS

UNKNOWN AT TIME OF PREDICTION

↓

TARGET

The model must respect this boundary.

Otherwise, it may appear remarkably accurate during testing but be impossible to use in reality.

Time determines what is an input

Suppose the target is:

electricity demand at 18:00 tomorrow.

At 12:00 today, we might know:

historical demand,
weather forecast,
calendar information.

We do not yet know:

actual temperature tomorrow at 18:00,
actual demand tomorrow at 17:30.

Those future observations cannot be used unless the real service will have them before making the prediction.

Forecasting requires careful timestamps

Suppose we have a historical dataset containing:

actual temperature

and:

actual electricity demand.

A model trained to predict tomorrow's demand might use actual tomorrow temperature.

Historically, that produces excellent predictions.

But in real operation, tomorrow's actual temperature is not available yet.

Only:

forecast temperature

is available.

So the correct training input may need to be:

weather forecast available at the forecast origin

rather than:

weather that eventually occurred.

This distinction can make a large difference.

Inputs are representations of reality

Recall from Module 2:

data is not reality itself.

The inputs a model receives are representations produced by:

sensors,
categories,
measurement systems,
databases.

Suppose we provide:

temperature = 20°C.

That number may contain:

measurement error,
spatial aggregation,
temporal aggregation.

The model only sees the representation.

Models cannot use information they do not receive

Suppose a medical model receives:

age,
symptoms,
blood pressure.

But an important factor is:

a genetic condition not included in the dataset.

The model cannot directly use that information.

It may sometimes infer related patterns from other features.

But fundamentally:

A model's view of the world is limited by its inputs.

What we exclude matters

Imagine predicting house prices.

We include:

size,
age.

But exclude:

location.

The model may perform poorly because location contains substantial predictive information.

Now include:

exact postcode.

Performance may improve dramatically.

Changing the input representation changes what the model can learn.

More inputs are not automatically better

It can be tempting to think:

Give the model every variable available.

But more features can create problems.

They may be:

irrelevant,
noisy,
duplicated,
expensive to collect,
unavailable in deployment.

More information can help.

More columns do not automatically mean more useful information.

Irrelevant inputs

Suppose we want to predict electricity demand.

Inputs include:

temperature,
time,
day.

Then we add:

random number generated for each observation.

This new feature contains no useful information about demand.

A good learning algorithm may learn to ignore it.

But with limited data or complex models, irrelevant features can still contribute to overfitting.

Redundant inputs

Suppose a model receives:

temperature in Celsius

and:

temperature in Fahrenheit.

These are not independent pieces of information.

One is exactly derived from the other.

Adding both does not really add knowledge.

Inputs can contain redundancy.

Correlated features

Real features are often related without being identical.

For example:

house size,
number of bedrooms.

Large houses tend to have more bedrooms.

Both may still provide useful information.

But correlated features affect how we interpret model parameters and feature importance.

Inputs can be numerical

Examples include:

temperature,
age,
income,
speed,
demand.

These may be:

continuous,
discrete.

The model receives numerical values directly or after preprocessing.

Inputs can be categorical

Suppose the input includes:

day of week.

Possible categories are:

Monday,
Tuesday,
Wednesday,
...

Or:

building type

may be:

residential,
commercial,
industrial.

Machine-learning systems need some way of representing these categories numerically.

We will see later that representation matters.

Inputs can be text

Suppose the task is:

Predict whether an email is spam.

The input may be the full email:

"Congratulations! You have won..."

Text must be transformed into a numerical representation that the model can process.

Modern language models do this using:

tokens,
embeddings.

We will explore those later.

Inputs can be images

Suppose the task is:

Identify whether an image contains a pedestrian.

The input is an image.

But to a computer, the image is ultimately represented as arrays of numbers.

Each pixel may contain values describing:

brightness,
colour.

So:

IMAGE

becomes:

NUMERICAL ARRAY

before being processed by the model.

Inputs can be sound

Audio can be represented as a sequence of numerical measurements through time.

For example:

air pressure amplitude at successive moments.

A speech-recognition model receives this signal and predicts:

text.

So:

SOUND WAVE

↓

NUMERICAL REPRESENTATION

↓

MODEL

↓

WORDS

Almost anything can become an input

Machine learning works with numerical representations.

So:

text,
images,
sound,
location,
movement,
network traffic

can all become model inputs once represented numerically.

This is one reason AI can operate across so many domains.

Outputs can be numerical

A numerical output gives us a regression problem.

Examples:

house price = €420,000

temperature = 18.4°C

demand = 40.2 GW

journey time = 37 minutes

The output lies on a numerical scale.

Outputs can be categories

A categorical output creates a classification problem.

Examples:

spam / not spam

cat / dog / fox

fraud / legitimate

healthy / faulty

The model predicts which class best fits the input.

Outputs can be probabilities

A classification model may output:

P(Fraud) = 0.82

rather than simply:

Fraud.

This preserves uncertainty.

The service can then choose how to act on the probability.

Outputs can be distributions

A forecasting model may produce:

expected demand = 40 GW

with a full probability distribution around that value.

This allows us to ask:

how uncertain is the prediction?
what is the tail probability?
how much reserve is needed?

A distribution contains more information than a point estimate.

Outputs can be sequences

Suppose a speech-recognition model receives audio and produces:

a sequence of words.

A translation model receives:

English sentence

and outputs:

French sentence.

A language model receives:

context

and produces:

a sequence of tokens.

These are sequence-to-sequence prediction problems.

Outputs can be images

A generative model might receive:

text prompt

and output:

an image.

So:

TEXT INPUT

↓

MODEL

↓

IMAGE OUTPUT

The output is high-dimensional rather than one simple number.

Outputs can be actions

An autonomous system might map:

sensor information

to:

steering command.

For example:

camera + velocity + road state

↓

model

↓

turn steering wheel 3 degrees left

Now the output directly affects the physical world.

This begins to blur the line between:

prediction

and:

decision-making.

Not every model output is a prediction target

Suppose an AI predicts:

pedestrian probability = 80%.

Then a control system outputs:

brake = yes.

The first is a predictive output.

The second is a decision.

Keeping these layers separate is useful.

Conceptually:

INPUTS

↓

PREDICTION MODEL

↓

PROBABILITY

↓

DECISION RULE

↓

ACTION

Outputs should represent what we care about

Suppose a school wants to predict:

student success.

What should the output be?

Possible labels include:

exam grade,
graduation,
employment,
student wellbeing.

These are not interchangeable.

The chosen output defines what "success" means to the model.

This is not merely a technical decision.

A badly chosen target produces a badly designed system

Suppose a hospital wants to improve health.

It trains a model to predict:

healthcare spending.

Perhaps spending correlates with health needs.

But spending is also affected by:

access,
insurance,
treatment availability.

If spending is used as a proxy for health need, the model may systematically underestimate people who historically received less care.

The model may predict its target accurately while failing the underlying service objective.

Proxy outputs

Sometimes the thing we care about cannot be measured directly.

So we choose a proxy.

For example:

engagement

may proxy:

user satisfaction.

exam score

may proxy:

understanding.

salary

may proxy:

career success.

Proxies can be useful.

But:

The model will learn the proxy we give it, not the deeper concept we hope the proxy represents.

Labels come from measurement systems

Suppose we train an AI to predict:

fraud.

How is fraud labelled?

Perhaps:

customer dispute,
investigator decision,
legal conviction.

Each creates a different dataset.

The target is therefore itself the result of:

measurement and institutional processes.

Labels can contain error

Suppose human moderators label comments:

toxic

or:

not toxic.

Different moderators may disagree.

The output labels used during training therefore contain uncertainty.

The model is learning from an imperfect representation of the target.

Labels can contain historical bias

Suppose an employment model predicts:

high performer

based on historical manager evaluations.

If those evaluations contained bias, the target contains that history.

The model may faithfully learn the biased label.

This is why:

high predictive accuracy

does not automatically imply:

fair prediction.

The output determines the learning objective

Suppose:

Y = price.

The model learns to predict price.

Suppose instead:

Y = probability of price exceeding €100.

Now it learns something different.

Same world.

Different target.

The model learns whatever target we define.

Prediction horizons change the output

Suppose:

Y = machine failure.

We need to specify:

within what period?

For example:

failure within 1 hour

failure within 24 hours

failure within 1 year

These are different outputs.

A machine might have:

low one-hour risk

but:

high one-year risk.

Spatial scale changes the output too

Suppose:

Y = electricity demand.

At what scale?

household,
neighbourhood,
city,
country.

Each target behaves differently.

Household demand may be highly variable.

National demand may be much smoother.

The output definition changes the prediction difficulty.

Resolution matters

Suppose we predict temperature.

Output could be:

nearest 1°C

or:

nearest 0.01°C.

The second is a much more demanding prediction problem.

Likewise, an image model might identify:

there is a pedestrian

or:

exact pixel-level pedestrian location.

The desired output resolution determines model requirements.

One input can produce many outputs

Suppose an autonomous vehicle receives one camera image.

It may predict:

pedestrians,
vehicles,
lane boundaries,
traffic lights,
depth.

This is multi-task learning.

One representation supports several outputs.

Multiple outputs may help learning

Suppose a model learns to predict:

pedestrian class,
pedestrian location,
pedestrian movement.

These tasks share information.

Learning them together can sometimes improve performance.

The model develops a richer representation.

Outputs can depend on each other

Suppose we forecast:

temperature,
humidity,
rainfall.

These outputs are related.

Predicting them jointly may preserve relationships that would be lost if each were modelled independently.

This creates multivariate prediction.

Inputs themselves can be uncertain

Suppose a model receives:

temperature = 20°C.

But the sensor has uncertainty:

±2°C.

A simple model may treat the input as exact.

A more sophisticated model might represent:

input uncertainty.

This can affect output uncertainty.

Missing inputs

Suppose the temperature sensor goes offline.

Does the model have to stop?

Not necessarily.

Possible approaches include:

imputation,
alternative sensors,
model-based estimation.

But the output uncertainty should usually increase.

This connects back to Module 2.

Input quality affects output quality

A model cannot completely escape its evidence.

Conceptually:

NOISY INPUT

↓

MODEL

↓

UNCERTAIN OUTPUT

A powerful model may handle noise well.

But it cannot create information that was never available.

Garbage in, garbage out

The phrase:

Garbage in, garbage out

captures part of this idea.

If inputs are:

wrong,
biased,
badly measured,

the predictions may also be poor.

But the phrase can be oversimplified.

Modern models can:

detect noise,
infer missing information,
combine signals.

The deeper principle is:

Output quality is constrained by the information contained in the inputs and the assumptions made by the model.

Information can be compressed

Suppose we have one year of raw electricity measurements.

Do we need to supply every historical value?

Perhaps not.

We may compute:

recent average,
trend,
hour of day,
previous-day demand.

These are compressed representations of the history.

Feature engineering often involves deciding what information should be preserved.

Modern AI often learns the representation itself

Traditional machine learning may require humans to create features manually.

For example:

image → edges → shapes → classifier.

Deep learning can learn internal representations directly from raw inputs.

For example:

pixels → neural network → learned representation → output.

This is one of the major developments that enabled modern AI.

Raw input is not necessarily truly raw

Even so-called raw data usually has already undergone processing.

An image has been:

captured by a sensor,
digitised,
resized,
compressed.

Text has been:

written,
encoded,
tokenised.

So every input representation contains design decisions.

There is no completely unmediated access to reality.

Inputs determine what distinctions the model can make

Suppose two situations produce exactly the same input vector.

The model receives:

x₁ = x₂.

Then a deterministic model must produce:

f(x₁) = f(x₂).

It cannot distinguish the cases.

If the real outcomes differ because of information absent from x, the model cannot perfectly predict both.

This is a fundamental limit.

Unobserved variables create irreducible uncertainty

Suppose customer demand depends partly on:

whether the customer unexpectedly invites ten friends over.

The model does not know this.

Even with perfect learning, some variation remains.

This is one source of aleatoric uncertainty.

Prediction is limited by the information available.

Better sensors can improve prediction

Suppose a robot predicts object motion from one camera.

Add:

radar,
lidar.

Now the inputs contain more information.

Prediction may improve.

This is sensor fusion.

Different input modalities provide complementary evidence.

Multimodal inputs

Modern AI systems may receive:

text,
image,
audio,
video.

For example:

image + question

↓

multimodal model

↓

text answer.

The model combines multiple forms of evidence.

This is a major direction in modern AI.

More modalities can reduce ambiguity

Suppose a model hears:

"bank".

The word could mean:

financial institution,
river bank.

Now supply an image of:

a river.

The ambiguity is reduced.

Additional modalities can provide context.

But modalities can conflict

Suppose:

camera says pedestrian

but:

radar says no object.

A good system should represent the conflict rather than pretending both inputs agree.

More inputs can create richer uncertainty, not merely greater certainty.

Input selection can create fairness problems

Suppose a hiring model uses:

postcode.

Postcode may improve prediction.

But it may also proxy:

income,
ethnicity,
access to education.

An input can be statistically useful while socially problematic.

So feature selection is not only about accuracy.

Removing sensitive features may not remove sensitive information

Suppose we remove:

protected characteristic.

But retain:

postcode,
school,
language,
income.

These may still strongly reveal it.

Models can infer information from proxies.

This will become important in the fairness module.

Inputs can create privacy problems

A model may benefit from:

location,
browsing history,
health records.

But should the service collect them?

Prediction accuracy is not the only objective.

Data collection must also consider:

privacy,
consent,
proportionality.

Sometimes a slightly less accurate model using much less personal data may be a better service.

Information has a cost

Collecting inputs may require:

sensors,
storage,
bandwidth,
human effort.

So more predictive information can create higher system cost.

An intelligent service may need to ask:

Is this feature worth collecting?

Value of information

Suppose adding one expensive sensor reduces forecast error by:

0.01%.

Perhaps it is not worth it.

Another cheap sensor reduces error by:

20%.

That may be highly valuable.

The value of an input depends on how much it improves the decisions the prediction supports.

The best feature is not necessarily worth having

Suppose exact household occupancy dramatically improves energy forecasting.

But obtaining it would require intrusive surveillance.

The feature may be statistically valuable.

The service may still decide not to collect it.

Technical value and social legitimacy are different.

Input design is part of service design

A prediction system must decide:

what information to collect,
at what resolution,
how often,
for how long,
from whom.

Those choices create:

cost,
privacy implications,
measurement bias.

The machine-learning model comes after these design decisions.

Outputs also create consequences

Suppose a system predicts:

risk score = 0.82.

Who sees that score?

What happens next?

Does it trigger:

human review,
automatic rejection,
additional testing?

The output interface influences how predictions become decisions.

A model score is not automatically a probability

Some machine-learning models produce numerical scores.

For example:

score = 4.7.

That may indicate stronger evidence for one class.

But unless it is calibrated appropriately, it should not automatically be interpreted as:

47% probability.

The meaning of the output matters.

Scores, probabilities and decisions

A service might contain three distinct layers:

MODEL SCORE

↓

CALIBRATION

↓

PROBABILITY

↓

THRESHOLD / DECISION RULE

↓

ACTION

Keeping these layers separate improves clarity.

Thresholds transform outputs

Suppose:

P(Fraud) = 0.43.

The service uses threshold:

0.50.

Then:

not fraud.

Change threshold to:

0.30.

Now:

fraud.

The model prediction did not change.

The decision rule changed.

This is another reason prediction and decision should not be conflated.

Different services can use the same model differently

Suppose one fraud model produces probabilities.

Low-risk service

Threshold:

90%

because false positives are costly.

High-security service

Threshold:

20%

because missing fraud is extremely costly.

Same prediction model.

Different decision design.

Outputs should preserve uncertainty when useful

Suppose a model predicts:

house price = €500,000.

A user might make a large financial decision based on it.

It may be more useful to output:

Expected price: €500,000

Likely range: €450,000–€560,000

The second output communicates uncertainty.

Interfaces can destroy useful information

A model may internally know:

Class A: 51%

Class B: 49%.

The interface shows:

CLASS A.

The user may assume high certainty.

The information loss occurred after prediction.

Good system design needs to consider how model outputs are communicated.

Humans need outputs they can act on

A full probability distribution may be mathematically ideal.

But a driver cannot inspect a 10,000-dimensional distribution while approaching a junction.

The final interface may need to transform complexity into:

BRAKE.

The challenge is to simplify appropriately without hiding important uncertainty.

Different users need different outputs

Consider an electricity forecast.

System operator

May want:

full distribution,
tail probabilities.

Analyst

May want:

expected value,
prediction interval.

Household consumer

May simply want:

Best time to run appliance: 14:00–16:00.

The underlying model can support several output representations.

Prediction outputs can become inputs elsewhere

Modern systems often contain chains of models.

For example:

weather forecast

becomes input to:

electricity demand forecast.

That prediction becomes input to:

grid optimisation.

So:

MODEL A OUTPUT

↓

MODEL B INPUT

↓

MODEL C INPUT

Uncertainty can propagate through the whole system.

Errors can propagate too

Suppose the weather forecast is wrong.

The demand model may then be wrong.

The optimisation may make the wrong scheduling decision.

A system of accurate individual models can still produce poor end-to-end performance if uncertainty is not propagated properly.

Prediction pipelines

A real AI system may therefore look like:

SENSORS

↓

DATA CLEANING

↓

FEATURE EXTRACTION

↓

MODEL

↓

CALIBRATION

↓

DECISION SYSTEM

↓

ACTION

The machine-learning model is only one part of the service.

Inputs and outputs can form feedback loops

Suppose a recommendation model predicts:

what the user will click.

Its output determines:

what content the user sees.

The user's response becomes:

new input data.

So:

INPUTS

↓

PREDICTION

↓

OUTPUT

↓

ACTION

↓

USER RESPONSE

↓

NEW INPUTS

The system becomes reflexive.

The output can alter future inputs

This is one of the most important ideas in deployed AI.

A credit model outputs:

reject loan.

The customer receives no loan.

So future training data contains:

no repayment outcome for that customer.

The output changed what evidence becomes available later.

Prediction systems can shape their own datasets.

Inputs are state; outputs can become actions

In dynamic systems, we can think of:

current state

as input.

The model predicts:

future state.

The decision system chooses:

action.

The action changes the state.

So:

STATE

↓

PREDICTION

↓

DECISION

↓

ACTION

↓

NEW STATE

This will become central in later modules.

A state vector is a structured input

Suppose a robot's state is:

x = [position, velocity, battery, orientation].

That state vector becomes the input to prediction or control models.

It represents:

Everything about the current system that we think matters for what happens next.

The quality of the state representation determines what the model can predict.

Observation is not always input state

Raw observations may include:

pixels,
radar readings,
GPS.

The system may first infer a state:

OBSERVATIONS

↓

STATE ESTIMATION

↓

STATE VECTOR

Then the state becomes the input to:

prediction and decision.

This is a more sophisticated architecture than feeding every raw observation directly into every downstream process.

Input representation can make learning easier

Suppose we want to predict the motion of a vehicle.

Input A:

two photographs

Input B:

estimated position + velocity.

Both may contain enough information.

But Input B makes the dynamical relationship much easier to learn.

Good representations expose useful structure.

Deep learning learns hidden features

A neural network may receive raw pixels and internally construct representations corresponding to:

edges,
shapes,
objects.

These hidden internal variables are often called:

representations or latent features.

The model effectively creates its own intermediate inputs.

Representation learning changed AI

Traditional machine learning often depended heavily on humans deciding:

Which features should we compute?

Deep learning allowed models to learn many useful features automatically.

This was a major shift.

Instead of:

RAW DATA

↓

HUMAN-DESIGNED FEATURES

↓

MODEL

we increasingly have:

RAW DATA

↓

LEARNED REPRESENTATION

↓

MODEL OUTPUT.

But the outer inputs and outputs are still designed by humans

Even if a neural network learns its own internal features, people still decide:

what data enters,
what target is provided,
what loss function scores the output.

The learning process is not free from design choices.

Inputs define evidence; outputs define success

This is perhaps the most important conceptual summary.

The inputs define:

What evidence is the system allowed to use?

The outputs define:

What question is the system being asked to answer?

Later, the loss function defines:

What counts as being wrong?

Together, these decisions create the learning problem.

Ask input questions

When examining a machine-learning system, ask:

What information enters the model?
When is it available?
How was it measured?
Is anything important missing?
Are some features redundant?
Are some features proxies for sensitive characteristics?
Could collecting them create privacy problems?
Is the training input representation the same as deployment?
Are inputs noisy or uncertain?
Is there data leakage?

Ask output questions

Also ask:

What exactly is the target?
Who defined it?
Is it directly measured or a proxy?
Does it contain historical bias?
What time horizon does it represent?
What spatial scale?
Is the output a number, class, probability or distribution?
How will that output be used?
Does the interface preserve uncertainty?
Does the output trigger a decision or action?

These questions often matter more than which algorithm is chosen.

The central idea

Every prediction problem can be understood through a basic mapping:

INPUTS

↓

MODEL

↓

OUTPUT

But behind that simple structure lie major design decisions.

Inputs determine:

what the model knows.

Outputs determine:

what the model is asked to predict.

So the richer framework is:

REALITY

↓

MEASUREMENT

↓

INPUT REPRESENTATION

↓

MODEL

↓

OUTPUT REPRESENTATION

↓

DECISION

The model does not see reality directly.

It sees the inputs we provide.

And it does not discover automatically what society cares about.

It learns the output we define.

Before asking whether a model is intelligent, ask what information it receives and what question we have asked it to answer.

With that structure established, we can now look at the two most fundamental families of supervised prediction.

The first deals with numerical outputs.

That is regression.