We have now separated four ideas that are often blurred together:

prediction
objectives
constraints
decisions

Optimisation is what brings the last three together.

An optimisation problem asks:

Given what we are trying to achieve, and given the limits of the real world, what is the best action we can take?

That sounds simple.

It can become extremely powerful.

The basic optimisation problem

Suppose we have a set of possible actions (a).

We define an objective function:

[ U(a) ]

which tells us how desirable each action is.

We also define a feasible set:

[ \mathcal{F} ]

containing the actions that satisfy all constraints.

Then the optimisation problem is:

[ a^*

\arg\max_{a \in \mathcal{F}} U(a) ]

or, if we are minimising cost:

[ a^*

\arg\min_{a \in \mathcal{F}} C(a) ]

The optimiser searches through the feasible actions and identifies the one that performs best according to the objective.

This gives us a very general structure:

[ \boxed{ \text{best feasible action} } ]

That phrase captures a huge amount of engineering, economics and operations research.

A simple example

Imagine a delivery company has three vans and twenty packages.

It wants to minimise total driving time.

The objective might be:

[ \min \text{total journey time} ]

subject to constraints such as:

[ \text{packages assigned to van } i \leq \text{van capacity} ]

and:

[ \text{driver hours} \leq \text{legal working limit} ]

The optimiser does not ask:

What route looks sensible?

It asks:

Among all valid assignments and routes, which one gives the lowest objective value?

The same mathematical idea can appear in:

logistics
healthcare
energy
finance
manufacturing
transport
telecommunications
public services

Optimisation is not prediction

This distinction is worth repeating.

A prediction model might estimate:

[ \text{travel time on road } j ]

An optimiser then uses those predicted travel times to choose:

[ \text{which route should we take?} ]

Prediction produces information.

Optimisation uses information to choose an action.

For example:

[ \text{traffic data} \rightarrow \text{prediction model} \rightarrow \text{estimated travel times} \rightarrow \text{optimisation} \rightarrow \text{route} ]

These are different layers of the system.

Optimisation needs an objective

An optimiser cannot optimise "generally."

It must optimise something.

For example:

[ \min \text{cost} ]

or:

[ \max \text{profit} ]

or:

[ \min \text{emissions} ]

or:

[ \max \text{throughput} ]

or:

[ \min \text{waiting time} ]

The answer depends entirely on what we choose.

That means optimisation is never neutral.

The mathematics may be precise.

The objective is still a design choice.

Optimisation also needs constraints

Without constraints, an optimiser can return absurd answers.

Suppose we want to maximise electricity generation revenue:

[ \max \sum_g p_g P_g ]

If there are no generator limits, the optimiser may simply choose:

[ P_g \rightarrow \infty ]

That is mathematically excellent and physically impossible.

So we add:

[ 0 \leq P_g \leq P_g^{\max} ]

The objective tells the optimiser where it wants to go.

The constraints stop it leaving reality.

Prediction + optimisation

Many modern AI systems combine these ideas.

Suppose a hospital predicts:

[ P(\text{benefit from treatment} \mid x_i) ]

for each patient (i).

An optimiser might then choose which patients receive treatment:

[ \max \sum_i x_i P_i(\text{benefit}) ]

subject to:

[ \sum_i x_i \leq 10 ]

where:

[ x_i

\begin{cases} 1 & \text{if patient } i \text{ receives treatment}\ 0 & \text{otherwise} \end{cases} ]

Now we have combined:

prediction
objective
constraints
decision

The model estimates benefit.

The optimiser allocates scarce capacity.

That is much closer to a real decision system.

Continuous and discrete decisions

Optimisation problems can involve different kinds of decisions.

Some decisions are continuous.

For example:

How much power should Generator A produce?

We might allow:

[ P_A = 347.2 \text{ MW} ]

Any value within a range may be possible.

Other decisions are discrete.

For example:

Should this generator be turned on?

Then:

[ u_A \in {0,1} ]

where:

[ u_A = 1 ]

means on,

and:

[ u_A = 0 ]

means off.

Discrete decisions often make optimisation much harder.

Linear optimisation

One important family of optimisation problems is linear programming.

Suppose the objective is:

[ \min c_1x_1 + c_2x_2 ]

subject to:

[ a_{11}x_1 + a_{12}x_2 \leq b_1 ]

[ a_{21}x_1 + a_{22}x_2 \leq b_2 ]

and:

[ x_1,x_2 \geq 0 ]

Everything is linear.

This structure is powerful because linear programmes can often be solved very efficiently even with large numbers of variables.

They appear in:

transport
scheduling
supply chains
electricity dispatch
production planning

Nonlinear optimisation

Real systems are not always linear.

Suppose cost depends on output as:

[ C(P) = aP^2 + bP + c ]

Now the objective is nonlinear.

Or suppose network physics create nonlinear relationships.

The optimisation problem may become:

[ \min C(x) ]

subject to:

[ g(x) = 0 ]

[ h(x) \leq 0 ]

where (C), (g) or (h) may be nonlinear.

Nonlinear optimisation can be much more difficult.

Convex optimisation

A particularly useful class of problems is convex optimisation.

Informally, a convex problem has a structure where:

any local optimum is also a global optimum.

This is extremely valuable.

Imagine standing in a bowl-shaped landscape.

Wherever you walk downhill, you eventually reach the same lowest point.

That is roughly the intuition behind convexity.

By contrast, a non-convex problem may look like a mountain range with many valleys.

You can walk downhill and end up in a valley that is good, but not the best.

Local and global optima

Suppose an objective function has several peaks.

An optimiser might find:

[ a_{\text{local}} ]

where small changes make things worse.

That is a local optimum.

But somewhere else there may be:

[ a_{\text{global}} ]

with a better objective value.

The global optimum is the best solution across the entire feasible space.

This distinction matters in:

neural-network training
engineering design
scheduling
market optimisation

Not every optimisation algorithm guarantees the global optimum.

Optimisation can be computationally difficult

Some decision spaces are enormous.

Imagine scheduling 100 workers across hundreds of tasks.

The number of possible schedules can become astronomical.

Or imagine deciding which of 1,000 generators should be on or off.

If each has two states:

[ 2^{1000} ]

possible combinations exist.

That number is unimaginably large.

The optimiser cannot necessarily try every possibility.

Instead, it uses mathematical structure and algorithms to search intelligently.

Mixed-integer optimisation

Problems containing both continuous and discrete decisions are often formulated as mixed-integer programmes.

For example:

[ P_g \geq 0 ]

might represent generator output,

while:

[ u_g \in {0,1} ]

represents whether the generator is running.

The system might solve:

[ \min \sum_g C_g(P_g,u_g) ]

subject to:

demand balance
generator limits
ramp rates
network constraints
minimum operating times

This is the basic flavour of many scheduling problems in electricity systems.

The important lesson is not the exact mathematics.

It is that optimisation can encode very rich decision logic.

Optimisation in electricity systems

Electricity systems are full of optimisation problems.

Suppose several generators can supply demand.

Generator A costs:

[ £30/\text{MWh} ]

Generator B costs:

[ £50/\text{MWh} ]

Generator C costs:

[ £80/\text{MWh} ]

Demand is:

[ 150 \text{ MW} ]

Each generator can produce up to:

[ 100 \text{ MW} ]

If our objective is:

[ \min \text{generation cost} ]

the optimiser might choose:

[ P_A = 100 ]

[ P_B = 50 ]

[ P_C = 0 ]

because that is the cheapest feasible dispatch.

Add a network constraint

Now suppose Generator A is located behind a transmission line that can carry only:

[ 60 \text{ MW} ]

Then:

[ P_A \leq 60 ]

The new optimum might be:

[ P_A = 60 ]

[ P_B = 90 ]

[ P_C = 0 ]

Same objective.

Different constraints.

Different optimum.

This is why system structure matters so much.

Change the objective

Now suppose the system also cares about emissions.

Generator A is cheap but highly polluting.

Generator C is expensive but zero-carbon.

We might define:

[ J

\alpha \times \text{cost} + \beta \times \text{emissions} ]

As (\beta) increases, the optimiser may shift generation toward Generator C.

Again:

same physical system,

different objective,

different optimum.

Optimisation exposes trade-offs

This is one of the most useful things optimisation can do.

It forces trade-offs into the open.

Instead of saying:

We want cheap, clean and reliable electricity.

we have to ask:

How much do we value each one?

Or:

Which requirements should be absolute constraints?

This can make policy choices much more explicit.

But it does not remove those choices.

Optimisation versus heuristics

Not every decision problem is solved with a formal optimiser.

Sometimes we use a heuristic.

A heuristic is a practical rule that usually works reasonably well.

For example:

Always serve the closest ambulance.

or:

Dispatch the cheapest generator first.

or:

Treat the most urgent patient first.

These rules can be simple and fast.

But they may not produce the best system-wide result.

Formal optimisation tries to search more systematically for better decisions.

Why use heuristics?

Because optimisation itself has costs.

A full optimisation model might require:

detailed data
computation
accurate system models
time
specialist implementation

In some situations, a simple rule is better.

If a decision must be made in milliseconds, a slightly worse answer available immediately may be more useful than a mathematically perfect answer delivered too late.

So real systems often trade off:

[ \text{solution quality} ]

against:

[ \text{computation time} ]

Approximate optimisation

Sometimes we deliberately accept a solution that is:

good enough.

Suppose the true global optimum has value:

[ 100 ]

and an algorithm finds:

[ 99.8 ]

in one second.

Finding exactly (100) might take six hours.

For many applications, the approximate solution is preferable.

This becomes especially important in real-time systems.

Optimisation under uncertainty

So far, we have often written optimisation as though all inputs are known.

But real systems contain uncertainty.

Suppose tomorrow's electricity demand is predicted to be:

[ 42 \text{ GW} ]

but could actually be:

[ 39 \text{ GW} ]

or:

[ 45 \text{ GW} ]

Then optimising only around:

[ 42 ]

may create a fragile plan.

We need to account for uncertainty.

Expected-value optimisation

One simple approach is to optimise the expected outcome.

Suppose demand (D) is uncertain.

We might choose:

[ a^*

\arg\min_a \mathbb{E}[C(a,D)] ]

The optimiser considers possible outcomes and their probabilities.

This is better than pretending the future is certain.

But expectation is not always enough.

Rare events matter

Suppose two plans have:

Plan A

Low average cost, but a small probability of catastrophic failure.

Plan B

Slightly higher average cost, but much more resilient.

Expected cost alone may favour Plan A.

Society may prefer Plan B.

This is why optimisation may need to consider:

risk
worst cases
tail events
resilience

not simply average outcomes.

Robust optimisation

Robust optimisation asks:

What decision performs reasonably well across a range of possible conditions?

Instead of optimising for one predicted future:

[ D = 42 ]

we might consider:

[ D \in [39,45] ]

and find a solution that remains feasible across that range.

The result may be less efficient in the most likely case.

But more resilient when reality surprises us.

Optimising a changing system

Many real systems are dynamic.

A decision today affects tomorrow.

Suppose a battery has state of charge:

[ E_t ]

and we choose discharge:

[ P_t ]

Then:

[ E_{t+1}

E_t

P_t \Delta t ]

If we discharge everything now, we cannot discharge later.

So we may need to optimise across time:

[ \min \sum_{t=1}^{T} C_t(a_t) ]

subject to state dynamics.

This is no longer just:

What is best now?

It becomes:

What sequence of actions is best over time?

Dynamic optimisation

This type of problem appears in:

battery scheduling
reservoir management
investment
inventory
robotics
finance
maintenance

The system has a state.

Actions change the state.

The future depends partly on what we do now.

That makes optimisation much richer.

Receding-horizon control

One common strategy is:

optimise over a future horizon
implement only the first decision
observe what actually happens
update the model
optimise again

This is known in control theory as model predictive control.

Conceptually:

[ \text{observe} \rightarrow \text{optimise} \rightarrow \text{act} \rightarrow \text{observe again} ]

This is interesting because it connects optimisation to feedback.

The optimiser is no longer solving the world once.

It is continuously updating decisions as the world changes.

Optimisation without feedback can fail

Imagine we create a perfect plan for tomorrow at midnight.

Then:

weather changes
demand changes
equipment fails
people behave differently

By midday, the plan may be wrong.

A static optimiser can still be excellent mathematically.

The system can still perform badly.

This is why real-world intelligence often requires:

[ \text{prediction} + \text{optimisation} + \text{feedback} ]

We will return to feedback later.

Optimisation changes behaviour

So far, we have talked as though the optimiser directly controls everything.

But many systems contain independent people and organisations.

Suppose a market designer wants lower electricity demand at peak times.

They cannot directly command every household.

Instead, they might change prices.

Households then optimise their own objectives.

Now the system contains many optimisers:

[ \text{household} \rightarrow \min \text{bill} ]

[ \text{generator} \rightarrow \max \text{profit} ]

[ \text{network} \rightarrow \min \text{congestion} ]

Their decisions interact.

This is much more complicated than one central optimisation problem.

One optimiser versus many

In a centrally controlled system, we might write:

[ \max W(a_1,\ldots,a_n) ]

and choose everyone’s actions together.

In a decentralised system, each participant may solve:

[ \max U_i(a_i) ]

independently.

The overall outcome emerges from their interaction.

This leads us toward:

game theory
mechanism design
markets
incentives

Optimisation is still present.

But there is no longer one obvious optimiser.

The optimiser does not know what is fair

Suppose we ask an optimisation system to minimise cost.

It finds a solution that saves:

[ £100 \text{ million} ]

Excellent.

But perhaps all of the savings come from cutting service to one vulnerable group.

The objective improves.

The social outcome may not.

The optimiser has done exactly what it was asked to do.

This is why fairness cannot simply be assumed to emerge from optimisation.

If fairness matters, it must appear somewhere in:

the objective
the constraints
the allocation rules
the institutional design

Efficiency is not the same as fairness

Imagine ten units of a scarce resource.

Two people want them.

Person A is willing to pay:

[ £100 ]

per unit.

Person B can afford only:

[ £10 ]

A revenue-maximising optimiser might allocate everything to A.

That may be optimal according to:

[ \max \text{revenue} ]

It does not follow that the allocation is fair.

Optimisation can tell us:

the best outcome according to this objective.

It cannot tell us:

whether this objective represents the values we actually want.

Optimisation can hide value choices

This is important because mathematical optimisation often looks objective.

A result might be presented as:

The optimal solution.

But we should always ask:

Optimal according to what objective?

Under what constraints?

Using whose data?

Over what time horizon?

With what assumptions?

Every optimum belongs to a model.

There is no context-free optimum.

A mathematically optimal system can still be badly designed

Suppose a transport model perfectly minimises average journey time.

But in doing so it:

eliminates rural services
increases pollution in poorer neighbourhoods
reduces resilience
makes travel impossible for people without cars

The optimiser may be flawless.

The design problem is not.

This is one of the central lessons of this module:

Optimisation is powerful, but it cannot rescue a badly specified system.

Optimisation is a tool, not a purpose

It is tempting to treat optimisation as the goal.

But optimisation is just a method.

Before optimising, we still need to ask:

What is the system for?
What should it optimise?
What must it never violate?
Who is affected?
What happens when resources are scarce?
How should competing objectives be handled?
How will the system adapt over time?

Only then does optimisation become meaningful.

AI and optimisation

Modern AI makes optimisation even more powerful.

AI can help estimate:

future demand
system state
user behaviour
equipment failure
likely outcomes

Optimisation can then use those estimates to choose actions.

We might have:

[ \text{data} \rightarrow \text{AI prediction} \rightarrow \text{optimisation} \rightarrow \text{decision} ]

This can produce extremely capable systems.

But every earlier question becomes more important, not less.

A more powerful optimiser can produce better decisions.

Or it can optimise the wrong thing faster and at larger scale.

Optimisation and search

There is also a deep connection between optimisation and many AI methods.

Training a neural network is itself an optimisation problem.

We choose parameters:

[ \theta ]

to minimise loss:

[ \theta^*

\arg\min_\theta L(\theta) ]

Gradient descent is an optimisation algorithm.

So AI often contains optimisation internally.

But that does not mean a trained AI system automatically knows how to optimise the external world.

These are different problems.

Training objective versus system objective

This distinction is especially important.

A language model may be trained to minimise:

[ \text{prediction loss} ]

But then deployed inside a customer-service system whose objective is:

[ \max \text{customer satisfaction} ]

The model’s training objective and the system’s operational objective are different.

A model can perform extremely well on its training objective while still being used badly in the real system.

Always distinguish:

What was the model trained to do?

from:

What is the deployed system trying to achieve?

From optimiser to system

We can now extend our framework again.

Prediction

What is likely to happen?

Objective

What are we trying to achieve?

Constraints

What is possible?

Optimisation

What is the best feasible action?

Decision

What action do we actually take?

And after that:

Outcome

What actually happens?

This gives us:

[ \text{Data} \rightarrow \text{Prediction} \rightarrow \text{Optimisation} \rightarrow \text{Decision} \rightarrow \text{Action} \rightarrow \text{Outcome} ]

But even this is incomplete.

Because the outcome changes the world.

And then we observe again.

Optimisation in a feedback loop

A more realistic system looks like:

[ \text{observe} \rightarrow \text{predict} \rightarrow \text{optimise} \rightarrow \text{act} \rightarrow \text{observe} \rightarrow \cdots ]

This is no longer a static decision.

It is a dynamic system.

That distinction will become increasingly important as we move from machine learning toward:

control
autonomous systems
markets
adaptive institutions
robotics

A practical checklist

When someone says:

We should optimise this system.

Ask:

[ \boxed{ a^*

\arg\max_{a \in \mathcal{F}} U(a) } ]

That simple expression contains several enormous assumptions:

what counts as good
what is possible
what information we have
what risks matter
whose interests count
what time horizon matters

Optimisation is one of the most powerful tools in modern engineering and AI.

But there is no such thing as:

the optimal decision

in isolation.

There is only:

the optimal decision for a particular objective, under particular constraints, using a particular model of the world.

So when someone says:

The AI found the optimal solution.

the right response is:

Optimal for what?