In the previous lesson, we made a crucial distinction:

Prediction tells us what might happen. Decision-making determines what we do about it.

But this immediately creates another question.

How do we decide what to do?

Suppose an AI system can predict the consequences of several possible actions.

Which action should it choose?

To answer that, the system needs some concept of what counts as a good outcome.

It needs an objective.

What is an objective?

An objective describes what we want a system to achieve.

In mathematical terms, we can imagine choosing an action (a) to maximise some objective:

[ a^* = \arg\max_a U(a) ]

where:

(a) is a possible action
(U(a)) measures how desirable that action is
(a^*) is the action we ultimately choose

Alternatively, we might formulate the problem as minimising something undesirable:

[ a^* = \arg\min_a C(a) ]

where (C(a)) represents some cost.

This looks simple.

It isn't.

Because the moment we define (U) or (C), we are making a statement about:

What does this system exist to achieve?

That question sits underneath almost every important decision system.

Predictions do not contain objectives

Consider our hospital example again.

Suppose a model predicts the probability that each patient will benefit from treatment:

Patient	Probability of benefit
A	82%
B	79%
C	76%
D	65%
E	61%

The model has given us useful information.

But what should the hospital optimise?

Perhaps:

Maximise the number of patients who benefit.

That sounds reasonable.

But perhaps the objective should instead be:

Minimise preventable deaths.

Or:

Maximise total years of healthy life.

Or:

Treat the most urgent patients first.

Or:

Minimise waiting time.

Or:

Ensure everyone has a fair opportunity to receive treatment.

Or some combination of these.

The predictions can be identical in every case.

The decisions can be completely different.

The objective determines what we do with the predictions.

The objective changes the answer

Imagine there are two patients and only one available treatment.

Patient A

90% probability of treatment success
expected benefit: 1 additional year of healthy life

Patient B

60% probability of treatment success
expected benefit: 20 additional years of healthy life

If our objective is:

Maximise the probability that treatment succeeds

we choose Patient A.

But suppose we calculate expected healthy-life benefit.

For Patient A:

[ 0.9 \times 1 = 0.9 ]

For Patient B:

[ 0.6 \times 20 = 12 ]

Now Patient B looks very different.

Same patients.

Same predictions.

Different objective.

Different decision.

This is one of the most important ideas in decision-making:

There is no such thing as the best decision without first asking: best according to what?

Objectives are everywhere

Many systems already have objectives, even when nobody explicitly calls them that.

A delivery company might try to:

minimise delivery time.

A factory might try to:

maximise production.

A power system might try to:

minimise the cost of supplying electricity.

A social-media platform might try to:

maximise engagement.

A hospital might try to:

maximise health outcomes.

A school might try to:

maximise examination performance.

A government department might try to:

minimise processing time.

Once we start building automated systems, these objectives become particularly important because computers require us to make them more explicit.

A human manager can operate with vague instructions such as:

Do what seems sensible.

An optimisation system needs something much closer to:

[ \text{minimise this} ]

or:

[ \text{maximise that} ]

That forces us to confront what we actually mean by success.

The metric becomes the target

There is a famous problem in system design:

When a measure becomes a target, people and systems begin optimising the measure rather than necessarily achieving the underlying purpose.

Suppose a call centre wants to provide excellent customer service.

That is difficult to measure directly.

So management chooses:

Average call duration

as a performance metric.

Perhaps shorter calls initially indicate efficient service.

The organisation therefore creates the objective:

[ \min \text{average call duration} ]

Employees respond rationally.

They shorten calls.

But now difficult customers may be rushed.

Problems may remain unresolved.

Customers may need to call back.

The measured objective improves.

The actual service gets worse.

The optimisation worked.

The objective was wrong.

Proxy objectives

This happens because the thing we really care about is often difficult to measure.

We therefore optimise a proxy.

For example:

What we actually care about	Possible proxy
Good education	Exam results
Good healthcare	Patients treated
Useful research	Publication count
Public safety	Arrest numbers
Customer satisfaction	Call duration
Valuable online content	Engagement
Productive employees	Hours online
Reliable electricity	Number of outages
Economic wellbeing	GDP

The proxy may contain useful information.

But:

[ \text{proxy} \neq \text{purpose} ]

The danger appears when the distinction is forgotten.

If an AI system becomes extremely effective at optimising the proxy, the problem can become worse rather than better.

AI can optimise the wrong thing extremely well

This is an important feature of advanced optimisation.

Suppose a recommendation algorithm has the objective:

[ \max \text{user engagement} ]

It becomes increasingly good at predicting:

Which piece of content will keep this person on the platform?

That might improve engagement dramatically.

But was engagement actually the purpose?

Perhaps what we wanted was:

useful information
meaningful social connection
entertainment
wellbeing
informed democratic participation

Engagement was simply easy to measure.

The AI may therefore become incredibly good at solving the problem we gave it while making the problem we actually cared about worse.

This is not necessarily an AI failure.

It may be an objective-design failure.

The difference between purpose and objective

It is useful to distinguish two concepts.

Purpose

Why does the system exist?

Objective

What measurable outcome are we asking the system to optimise?

Ideally:

[ \text{objective} ]

should be a good operational representation of:

[ \text{purpose} ]

But they can drift apart.

Consider a university.

Its purpose might include:

creating knowledge
educating students
developing independent thinkers
advancing science
serving society

But suppose institutional performance is measured primarily through:

publication counts
citation metrics
student satisfaction scores
rankings
grant income

Those measurements begin influencing behaviour.

Researchers respond to publication incentives.

Departments respond to rankings.

Administrators respond to satisfaction metrics.

Students respond to grading systems.

Eventually the organisation may become extremely effective at optimising its metrics without necessarily becoming better at fulfilling its original purpose.

The objective function has quietly replaced the mission.

One objective is rarely enough

Real systems usually care about several things simultaneously.

An electricity system might want:

low cost
high reliability
low emissions
network security
fairness
investment
resilience

A hospital might care about:

survival
quality of life
waiting times
fairness
cost
staff workload
capacity

A transport system might care about:

journey time
safety
emissions
accessibility
cost
congestion
reliability

These objectives can conflict.

Reducing one cost may increase another.

Improving efficiency may reduce resilience.

Maximising utilisation may eliminate spare capacity.

Prioritising one group may disadvantage another.

Decision-making therefore often involves multiple objectives.

Multi-objective optimisation

Suppose an electricity system cares about both cost and carbon emissions.

We might write:

[ J(a)

\alpha C(a) + \beta E(a) ]

where:

(C(a)) is financial cost
(E(a)) is emissions
(\alpha) determines how much importance we place on cost
(\beta) determines how much importance we place on emissions

The system then chooses:

[ a^* = \arg\min_a J(a) ]

But notice what has happened.

We have not eliminated the value judgement.

We have moved it into:

[ \alpha ]

and:

[ \beta ]

Who chooses those numbers?

Why should emissions receive one weight and cost another?

Would everyone choose the same weights?

Probably not.

The mathematics can make the trade-off explicit.

It cannot tell society what the trade-off ought to be.

There may be no single "best" answer

Suppose we have two possible electricity-system designs.

System A

lower cost
higher emissions

System B

higher cost
lower emissions

Which is better?

There is no purely mathematical answer unless we first specify how much we value each objective.

Now imagine:

System C

slightly higher cost
much lower emissions
much higher reliability

And:

System D

very low cost
low emissions
poor resilience during extreme weather

The decision becomes richer again.

Real systems live in these trade-offs.

AI does not make them disappear.

It may help us understand them much better.

Pareto improvements

One useful concept is a Pareto improvement.

Suppose we change a system and:

one objective improves
no other objective gets worse

That is relatively easy to justify.

But often improvements require trade-offs.

Perhaps:

[ \text{cost} \downarrow ]

while:

[ \text{reliability} \downarrow ]

Or:

[ \text{efficiency} \uparrow ]

while:

[ \text{resilience} \downarrow ]

Now we have to decide how much of one thing we are willing to exchange for another.

This is not simply a prediction problem.

It is a question about objectives and values.

Objectives can belong to different people

There is another complication.

Who is:

the system

in the first place?

Consider an electricity network.

The generator may want to:

[ \max \text{profit} ]

The household may want to:

[ \min \text{electricity bill} ]

The network operator may want to:

[ \min \text{congestion} ]

The government may want to:

[ \min \text{emissions} ]

The regulator may want to:

[ \max \text{consumer welfare} ]

The battery owner may want to:

[ \max \text{battery revenue} ]

These are not necessarily the same objective.

So when someone says:

We should use AI to optimise the electricity system.

we should immediately ask:

Optimise it for whom?

And:

Optimise what?

Those questions matter enormously.

Individual objectives and system objectives

This distinction appears throughout economics and system design.

Individual participants may each have their own objectives.

For participant (i):

[ \max U_i(a_i) ]

But the overall system may have some broader objective:

[ \max W(a_1,a_2,\ldots,a_n) ]

where (W) represents some measure of system-level welfare or performance.

These objectives may not naturally align.

A company may benefit from behaviour that imposes costs on others.

A household may respond rationally to a price signal in a way that creates network congestion.

A driver may choose the fastest route individually while thousands of drivers making the same choice create a traffic jam.

Everyone can behave rationally according to their own objective while the overall system performs badly.

This is why system design matters.

Objectives and incentives

Sometimes we cannot directly control what participants do.

Instead, we design incentives.

Suppose we want electricity consumers to reduce demand during periods of scarcity.

The system could send a higher price.

Consumers then decide whether reducing consumption serves their own objectives.

The system designer is therefore trying to create a relationship between:

[ \text{individual objective} ]

and:

[ \text{system objective} ]

Good system design tries to make desirable behaviour individually sensible.

This is a very different problem from simply predicting electricity demand.

It requires us to think about:

objectives
incentives
behaviour
feedback
allocation

We will return to these ideas later.

Objectives can create unintended behaviour

Whenever you create an objective, ask:

What behaviour would a highly capable optimiser produce if it took this objective literally?

Suppose a school rewards teachers entirely according to exam scores.

What behaviours might emerge?

Perhaps:

excellent teaching
teaching to the test
concentrating resources on borderline students
avoiding difficult pupils
narrowing the curriculum
manipulating assessments

The optimiser does not need to be an AI.

Humans respond to objectives too.

AI simply increases our ability to optimise at scale and speed.

That makes objective design more important, not less.

Constraints are not the same as objectives

There is another useful distinction.

Suppose an electricity system wants to minimise cost.

That is the objective:

[ \min C(a) ]

But it must also keep power lines within safe limits:

[ F_l \leq F_l^{\max} ]

That is a constraint.

The system is not necessarily trying to minimise power flow.

It simply cannot exceed a safe limit.

Similarly, a hospital might try to maximise patient benefit:

[ \max B(a) ]

subject to:

[ \text{beds used} \leq 10 ]

The ten-bed limit is not an objective.

It is a constraint.

This distinction will become important.

A decision problem often has the form:

[ \boxed{ \text{Choose the best feasible action} } ]

where:

best comes from the objective,

and:

feasible comes from the constraints.

Hard constraints and soft objectives

Sometimes certain requirements should not be traded away at all.

For example:

A bridge must not collapse.

It would be strange to write:

[ \text{minimise cost} + \text{some penalty for bridge collapse} ]

and allow the optimiser to decide that occasional collapse is economically worthwhile.

Instead, safety might be a hard constraint:

[ P(\text{structural failure}) < \epsilon ]

Similarly, an electricity network may require:

[ |F_l| \leq F_l^{\max} ]

regardless of whether violating that constraint would temporarily reduce cost.

This raises an important design question:

Which things should be objectives, and which should be constraints?

Should fairness be something we try to maximise?

Or should certain fairness requirements simply never be violated?

Should safety compete against profit?

Or should safety define the feasible operating region?

These are system-design choices.

Objectives can change over time

Real systems are also dynamic.

The correct objective at one moment may not be the correct objective at another.

During normal electricity-system operation, we might prioritise:

[ \text{cost efficiency} ]

During an emergency, the priority might become:

[ \text{system survival} ]

A hospital operating normally might focus on:

quality of care
waiting times
efficiency

During a mass-casualty event, triage priorities may change dramatically.

Objectives therefore exist within context.

A robust decision system needs to recognise that the world changes.

Short-term and long-term objectives

Another common failure occurs when we optimise the short term while damaging the long term.

A company could maximise this quarter's profit by:

cutting maintenance
reducing research
underinvesting in staff
exhausting natural resources

The immediate objective improves.

Long-term capability deteriorates.

Likewise, an electricity system could minimise today's cost while failing to invest in the capacity needed ten years from now.

A government could minimise current expenditure while allowing infrastructure to decay.

An AI system optimising a short-term metric can accelerate this problem dramatically.

So we must ask:

Over what time horizon are we optimising?

The objective:

[ \max U_t ]

may produce very different behaviour from:

[ \max \sum_{t=0}^{T} \gamma^t U_t ]

where future outcomes also matter.

The parameter (\gamma) represents how strongly we value future outcomes.

Again, apparently technical parameters can contain important assumptions about values.

What about future generations?

This becomes especially important in areas such as:

climate change
infrastructure
public debt
natural resources
biodiversity
energy investment

People who will live fifty years from now cannot participate in today's decision.

Yet today's objectives can dramatically affect them.

If our objective considers only immediate benefits:

[ U_{\text{today}} ]

we may make very different decisions than if we consider:

[ U_{\text{today}} + U_{\text{future}} ]

System design therefore forces us to ask:

Whose objectives count?

and:

When do they count?

These are not questions machine learning can answer for us.

The danger of saying "the AI decided"

This is why statements such as:

The AI decided who should receive treatment.

can be misleading.

The AI may have produced predictions.

But someone designed:

the objective
the decision rule
the constraints
the thresholds
the data
the allocation mechanism

The eventual action may be automated.

The values embedded in it did not appear from nowhere.

Whenever an automated system makes consequential decisions, we should therefore ask:

What objective is this system actually optimising?

That question is often more revealing than asking:

What AI model does it use?

A powerful model cannot rescue a bad objective

Imagine two systems.

System A

Uses a relatively simple predictive model but has:

a carefully designed objective
sensible constraints
fair allocation rules
good feedback
human oversight

System B

Uses the world's most advanced AI model but optimises:

[ \max \text{the wrong thing} ]

Which system would you rather trust?

Model sophistication is only one part of system quality.

A highly capable optimiser pointed at the wrong objective can simply help us reach the wrong destination faster.

From intelligence to purpose

This gives us a useful hierarchy.

Prediction asks:

What is likely to happen?

Decision-making asks:

What should we do?

Objectives ask:

What are we trying to achieve?

And system design asks:

How do we make the whole thing work when different people, objectives and constraints interact?

This is why artificial intelligence cannot be separated completely from the systems around it.

Better intelligence does not eliminate the need to decide what we value.

It makes that decision more important.

A simple framework

When you encounter an AI decision system, try asking these questions:

1. What is being predicted?

What information is the model producing?

2. What decision is being made?

What action follows?

3. What is the objective?

What counts as a good outcome?

4. Whose objective is it?

Who benefits when the metric improves?

5. Is the objective a proxy?

Does the measured target really represent the underlying purpose?

6. What are the constraints?

What is the system not allowed or physically able to do?

7. What trade-offs are being made?

What improves when the objective improves, and what might get worse?

8. What happens over time?

Does short-term optimisation damage long-term outcomes?

These questions often tell us more about the consequences of an AI system than the model architecture itself.

The key takeaway

An objective tells a decision system what counts as success.

That makes objective design one of the most consequential parts of any automated system.

[ \boxed{\text{Prediction estimates what may happen.}} ]

[ \boxed{\text{An objective defines what we want to happen.}} ]

[ \boxed{\text{Decision-making chooses what to do.}} ]

A system can have excellent data, excellent predictions and extremely powerful AI and still produce terrible outcomes if it is optimising the wrong objective.

So whenever someone proposes using AI to optimise something, the first question should not be:

How powerful is the model?

It should be:

What, exactly, are we asking it to optimise?

In the next lesson, we can build on this by examining what happens when objectives collide with scarcity, constraints and competing claims on limited resources.