Mean Squared Error

In the previous lesson, we introduced Mean Absolute Error.

The idea was simple:

Measure how far each prediction is from reality, ignore the direction of the error, and calculate the average.

If a prediction is wrong by 10 units, it contributes an error of 10.

If it is wrong by 20 units, it contributes an error of 20.

So an error twice as large receives exactly twice the penalty.

But this raises an important question:

Should a prediction that is twice as wrong always be considered only twice as bad?

Sometimes the answer is no.

In many systems, small prediction errors are relatively harmless while very large errors can have disproportionately serious consequences.

One way of reflecting this mathematically is to square the errors.

This gives us another of the most important measures in machine learning:

Mean Squared Error.

---

From absolute error to squared error

Suppose the actual value is:

$$y = 20$$

and our model predicts:

$$\hat{y} = 18$$

The error is:

$$y - \hat{y} = 2$$

With Mean Absolute Error, we take the absolute value:

$$|2| = 2$$

With Mean Squared Error, we instead square the error:

$$2^2 = 4$$

Now suppose the prediction is wrong by 4.

With absolute error:

$$|4| = 4$$

But with squared error:

$$4^2 = 16$$

The error became twice as large:

$$2 \rightarrow 4$$

but its squared penalty became four times as large:

$$4 \rightarrow 16$$

This is the central idea behind Mean Squared Error.

Large errors receive disproportionately large penalties.

---

Why squaring solves the sign problem

Remember the problem we encountered when calculating ordinary prediction errors.

Suppose our errors are:

$$+4,\ -4,\ +3,\ -3$$

If we simply average them:

$$\frac{4 - 4 + 3 - 3}{4} = 0$$

The positive and negative errors cancel.

Mean Absolute Error solves this by taking the absolute value.

Squaring provides another solution.

$$(+4)^2 = 16$$ $$(-4)^2 = 16$$ $$(+3)^2 = 9$$ $$(-3)^2 = 9$$

Once an error is squared, it is always positive.

So positive and negative errors can no longer cancel each other out.

---

Mean Squared Error

For a collection of predictions, we square each prediction error and then calculate their average.

This gives us Mean Squared Error, usually abbreviated to MSE.

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i-\hat{y}_i)^2$$

where:

• $n$ is the number of predictions;
• $y_i$ is the actual value;
• $\hat{y}_i$ is the predicted value.

The process is therefore:

Actual value → Prediction → Error → Square the error → Average

Or more simply:

Calculate how wrong every prediction was, square those errors, and take the average.

---

A simple example

Suppose we are predicting temperatures over five days.

Day	Actual	Predicted	Error	Squared error
Monday	18°C	20°C	-2	4
Tuesday	20°C	19°C	1	1
Wednesday	17°C	20°C	-3	9
Thursday	16°C	16°C	0	0
Friday	19°C	17°C	2	4

The total squared error is:

$$4 + 1 + 9 + 0 + 4 = 18$$

There are five predictions.

Therefore:

$$\text{MSE} = \frac{18}{5} = 3.6$$

So the model has:

MSE = 3.6

---

But what are the units?

There is something slightly strange about this result.

Our original predictions were temperatures measured in:

degrees Celsius

But because we squared the errors, MSE is technically measured in:

$$°C^2$$

That is not a particularly intuitive physical quantity.

If we predict house prices in pounds, MSE is measured in:

$$£^2$$

If we predict electricity demand in megawatts, MSE is measured in:

$$MW^2$$

This makes MSE less directly interpretable than Mean Absolute Error.

With MAE, we could say:

"Our predictions are about 1.6°C away from reality on average."

MSE does not have such a straightforward interpretation in the original units.

Its strength lies somewhere else.

It makes large errors matter much more.

---

The penalty grows quadratically

Consider errors of increasing size.

Error	Absolute error	Squared error
1	1	1
2	2	4
3	3	9
4	4	16
5	5	25
10	10	100
20	20	400
50	50	2,500

With Mean Absolute Error, the penalty grows in a straight line.

An error of 20 is twice as costly as an error of 10.

With squared error:

$$10^2 = 100$$

while:

$$20^2 = 400$$

An error twice as large receives four times the penalty.

An error three times as large receives:

$$3^2 = 9$$

times the penalty.

An error ten times as large receives:

$$10^2 = 100$$

times the penalty.

This is called a quadratic penalty.

---

MAE and MSE see mistakes differently

This difference becomes important when comparing models.

Imagine two models each make five predictions.

Model A

Its errors are:

$$4,\ 4,\ 4,\ 4,\ 4$$

Its Mean Absolute Error is:

$$\text{MAE}_A = 4$$

Its Mean Squared Error is:

$$\text{MSE}_A = \frac{16+16+16+16+16}{5} = 16$$

Now consider another model.

Model B

Its errors are:

$$0,\ 0,\ 0,\ 0,\ 20$$

Its Mean Absolute Error is:

$$\text{MAE}_B = \frac{20}{5} = 4$$

According to MAE:

Model A and Model B are equally good.

Both have:

$$\text{MAE}=4$$

But now calculate MSE.

For Model B:

$$\text{MSE}_B = \frac{0+0+0+0+400}{5} = 80$$

So:

$$\text{MSE}_A = 16$$

while:

$$\text{MSE}_B = 80$$

MSE strongly prefers Model A.

Why?

Because Model B made one very large mistake.

MAE says:

"Both models are wrong by 4 units on average."

MSE says:

"Yes, but Model B occasionally gets things very wrong, and I care a lot about that."

---

MSE is sensitive to outliers

This property makes MSE particularly sensitive to outliers.

An outlier is an observation that lies unusually far away from most of the data.

Suppose a model normally makes errors between 1 and 5 units.

Then suddenly it makes an error of:

$$50$$

That single observation contributes:

$$50^2 = 2500$$

to the squared error.

It may therefore have a very large influence on the final MSE.

This can be either a strength or a weakness.

---

When sensitivity to large errors is useful

Sometimes large errors are precisely what we want the model to avoid.

Consider electricity demand forecasting.

Suppose the electricity system expects demand of:

30 GW

but actual demand is:

30.1 GW

The forecast error is:

0.1 GW

That may be relatively easy to manage.

Now suppose the forecast predicts:

30 GW

but actual demand becomes:

36 GW

A 6 GW forecasting error is not merely a slightly larger version of the first mistake.

It could create a much more serious operational challenge.

The system may need:

• additional generation;
• reserve activation;
• battery discharge;
• demand response;
• electricity imports;
• emergency balancing actions.

If sufficiently serious, a large forecasting error could threaten system reliability.

In such a situation, we might deliberately want an evaluation metric that says:

Very large errors are especially bad.

MSE does exactly that.

---

When sensitivity to large errors is a problem

But suppose the large error wasn't really the model's fault.

Imagine we are predicting house prices.

Most houses sell for between:

£200,000 and £800,000

but one extraordinary mansion sells for:

£25 million.

The model predicts:

£5 million.

Its error is therefore:

$$£20,000,000$$

Squaring that error produces an enormous number.

That single property could dominate the MSE for the entire dataset.

The question then becomes:

Should one unusual case have so much influence over how we evaluate the model?

Perhaps yes.

Perhaps predicting £25 million properties correctly is extremely important.

But perhaps not.

Perhaps the model was designed primarily for ordinary residential properties and this observation belongs to a fundamentally different market.

MSE doesn't know.

It simply applies the rule we gave it:

Square the errors.

---

The metric does not understand the world

This is an important recurring idea.

Mean Squared Error has no concept of:

• electricity;
• houses;
• patients;
• money;
• weather;
• safety;
• fairness;
• reliability.

It knows only numbers.

If an error is:

$$e$$

the penalty is:

$$e^2$$

Whether that mathematical penalty corresponds to the real consequence of the error is something we have to decide.

The metric does not understand the system.

We do.

---

Why is MSE so common in machine learning?

Mean Squared Error is extremely common not only because it penalises large errors.

It also has useful mathematical properties.

The function:

$$e^2$$

is smooth.

There is a well-defined gradient almost everywhere we need it.

This matters because, as we saw earlier in the course, many machine-learning models learn using gradient descent.

The model repeatedly asks:

How should I change my parameters to reduce the loss?

If the loss function changes smoothly, we can calculate the direction in which the parameters should move.

Squared error works particularly well for this.

This is one reason it appears so frequently in:

• linear regression;
• neural-network training;
• forecasting;
• optimisation;
• statistical estimation.

The choice of loss function affects not only how we evaluate a model, but also how the model learns.

---

Loss function versus evaluation metric

This gives us an important distinction.

A loss function is used during training.

It tells the learning algorithm:

How bad was this prediction?

An evaluation metric is used to assess the resulting model.

It asks:

How well does this model perform?

Sometimes the same mathematical function is used for both.

For example, we might train a regression model by minimising MSE and then report MSE when evaluating it.

But this is not required.

A model could be trained using MSE and evaluated using:

• MAE;
• MSE;
• another metric;
• or several metrics simultaneously.

Training and evaluation are related, but they are not the same thing.

---

What minimising MSE encourages

Suppose a model repeatedly encounters large prediction errors.

Because those errors are squared, they contribute disproportionately to the loss.

The learning process therefore receives a strong incentive to reduce them.

We can think of the model as being told:

Small mistakes matter.

Large mistakes matter a lot.

This changes what the model learns to prioritise.

The choice between MAE and MSE is therefore not merely a choice about how to calculate a score afterwards.

If used as a training loss, it can change the model itself.

---

There is a statistical interpretation too

There is another reason squared error appears so often.

Suppose we assume that prediction errors behave approximately like a normal distribution.

That means most errors are relatively small, while increasingly large positive and negative errors become increasingly unlikely.

Under certain assumptions, finding the model that minimises squared error corresponds to finding the model that is most likely to have generated the observed data.

This connects machine learning to classical statistics.

It is one reason least-squares regression has such a central place in statistical modelling.

The mathematics is deeper than we need for now, but the important point is:

MSE isn't an arbitrary invention. It connects naturally to common assumptions about how random errors behave.

---

MSE versus MAE

We can now compare the two metrics directly.

Property	MAE	MSE
Error transformation	Absolute value	Square
Large errors	Linear penalty	Quadratic penalty
Sensitive to outliers	Less	More
Original units preserved	Yes	No
Easy physical interpretation	Very	Less
Strongly discourages large errors	No	Yes
Smooth for optimisation	Mostly	Yes

Neither metric is universally better.

They answer slightly different questions.

MAE asks:

How far away are my predictions, on average?

MSE asks something closer to:

How large are my errors if I particularly care about avoiding big mistakes?

---

Choosing between MAE and MSE

Suppose you are designing a forecasting system.

If an error twice as large should count roughly twice as much, MAE may be a sensible measure.

If an error twice as large should receive a much larger penalty, MSE may be more appropriate.

But there is no universal rule saying:

Machine learning should use MSE.

The right metric depends on the system.

Ask:

• What kinds of errors occur?
• How frequent are extreme errors?
• Are extreme observations genuine or erroneous?
• Are large errors particularly dangerous?
• Does the direction of error matter?
• What decisions will be made from the predictions?
• What are the real-world consequences?

Only then can we decide what "good prediction" actually means.

---

A subtle problem: the units are squared

We saw earlier that MSE does not retain the original units.

If we predict electricity demand in MW:

$$\text{MSE}$$

is measured in:

$$MW^2$$

That makes the number difficult to communicate.

Suppose:

$$\text{MSE}=40,000\ MW^2$$

What does that mean operationally?

It is not immediately obvious.

There is a simple mathematical trick we can use.

Take the square root:

$$\sqrt{40,000\ MW^2} = 200\ MW$$

This gives us a metric called:

Root Mean Squared Error, or RMSE.

RMSE restores the original units while retaining MSE's strong sensitivity to large errors.

We will encounter it frequently in real machine-learning and forecasting work.

---

The deeper lesson

At first glance, MAE and MSE appear to be minor mathematical variations.

One uses:

$$|e|$$

and the other uses:

$$e^2$$

But that tiny mathematical change encodes a very different idea about the world.

MAE says:

Every additional unit of error matters equally.

MSE says:

The larger the mistake becomes, the more severely each additional unit should be punished.

Neither statement is inherently correct.

They represent different assumptions about what kinds of errors matter.

And this brings us back to one of the central ideas of the course:

Metrics contain values.

When we choose an evaluation metric, we are not simply measuring a model.

We are defining what we mean by good.

---

The bigger idea

We now have two ways of evaluating numerical predictions:

Mean Absolute Error

$$\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} |y_i-\hat{y}_i|$$

Every additional unit of error receives the same additional penalty.

Mean Squared Error

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i-\hat{y}_i)^2$$

Large errors receive disproportionately larger penalties.

The difference looks small.

But it changes which model we might prefer.

That should make us suspicious whenever someone says:

"Model A performs better than Model B."

The immediate question should be:

Better according to what measure?

Because changing the measure can change the answer.

And when we move from predicting numbers to predicting categories — disease or no disease, fraud or legitimate, dangerous or safe — this question becomes even more important.

There, simply counting correct and incorrect predictions will not be enough.

We will need to distinguish between different kinds of mistakes.

That brings us next to:

Precision and Recall

When a model says "yes", how often is it right?

And of all the things it should have found, how many did it actually find?