Images, sound and language as data

So far, many of our examples have looked like conventional data.

Numbers in tables.

Temperatures.

Electricity demand.

Locations.

Counts.

But much of the information humans use to understand the world does not naturally arrive in neat rows and columns.

We see.

We hear.

We speak.

We read.

A photograph, a voice recording and a paragraph of text may look completely different from a spreadsheet, but to a computer they can all become data.

The crucial step is representation.

For a machine to process an image, sound or language, it must first be represented in a form that computation can operate on.

Ultimately, that means numbers.

This lesson explores how extremely rich human experiences such as vision, sound and language become numerical representations — and why those representations made modern artificial intelligence possible.

Computers operate on representations

Consider three things:

a photograph of a dog

someone saying the word "dog"

the written word "dog"

To a person, these can all refer to the same underlying concept.

But physically they are completely different.

A photograph consists of patterns of light.

Speech consists of pressure waves moving through air.

Written text consists of symbols.

Before a computer can process any of them, they must be transformed into numerical representations.

Conceptually:

WORLD

↓

MEASUREMENT / ENCODING

↓

NUMERICAL REPRESENTATION

↓

COMPUTATION

Different kinds of information require different transformations.

Images as data

Consider a digital photograph.

To a human, it might contain:

a person,
a bicycle,
a road,
a tree,
the sky.

A computer does not initially receive those concepts.

It receives numbers.

Pixels

A digital image is usually divided into a grid of small elements called pixels.

Imagine a tiny image containing:

4 × 4 pixels

It can be represented as a grid:


Pixel	Pixel	Pixel	Pixel
Pixel	Pixel	Pixel	Pixel
Pixel	Pixel	Pixel	Pixel
Pixel	Pixel	Pixel	Pixel

Each pixel stores information about the light detected at that position.

A grayscale image might represent brightness using a number such as:

0 → black

through to:

255 → white

So a tiny grayscale image might look numerically like:

12   14   18   25
15   40   90   31
18   75  220   45
20   32   55   28