Structured versus unstructured data

Not all data arrives in neat tables.

Some information fits naturally into rows and columns.

Other information comes as:

documents,
photographs,
audio recordings,
videos,
emails,
conversations,
web pages,
sensor logs.

This gives us a useful distinction between:

structured data

and:

unstructured data

The difference is not that one contains structure and the other contains none.

All information has some form of structure.

The distinction is really about how explicitly that structure has been defined in advance.

Structured data

Structured data follows a predefined organisation.

A simple example is a table:

Customer	Age	Region	Annual spend
A	34	Dublin	€4,200
B	51	Cork	€6,800
C	27	Galway	€3,100

Each column has a defined meaning.

Each row follows the same format.

We know that:

Age contains age values,
Region contains locations,
Annual spend contains monetary values.

The structure is explicit.

This makes the data relatively easy for computers to:

store,
filter,
sort,
aggregate,
compare,
analyse.

Schemas

The predefined organisation of a dataset is often described by a schema.

A schema might specify:

customer_id → text
age → integer
region → category
annual_spend → number

The schema tells the system what kind of information belongs in each field.

A database can then enforce rules such as:

age must be a number,
customer ID must be unique,
date must follow a particular format.

This consistency makes structured data powerful.

Databases are built around structure

Traditional databases are particularly good at managing structured information.

A company might have separate tables for:

customers,
orders,
products,
payments.

An order might contain fields such as:

Field	Example
Order ID	48321
Customer ID	C1042
Date	2026-08-06
Product	P207
Quantity	3
Price	€42

Because the structure is known, we can ask precise questions.

For example:

How many units of Product P207 were sold in July?

The database knows exactly which fields to examine.

Structured data is easy to query

Suppose we have one million transactions.

We can ask:

Show all purchases over €100 made in Dublin last month.

If the dataset is well structured, this is relatively straightforward.

We filter:

location = Dublin

then:

date = last month

then:

value > €100

The fields already tell us where each piece of information belongs.

This is one reason structured data has historically been so important for analytics.

Unstructured data

Now consider an email:

Hi, the delivery arrived this morning but two of the items were damaged. Could you send replacements before Friday?

Where is:

delivery_status

or:

number_of_damaged_items

or:

requested_resolution

stored?

Nowhere explicitly.

The information exists inside the text.

A human reader can infer:

delivery arrived = yes
damaged items = 2
requested action = replacement
deadline = Friday

But these fields were not defined in advance.

This is what we usually mean by unstructured data.

Examples of unstructured data

Common examples include:

emails,
reports,
books,
social-media posts,
photographs,
videos,
voice recordings,
meeting transcripts,
customer reviews,
medical notes.

These contain enormous amounts of useful information.

But their meaning is not already organised into fixed database columns.

Unstructured does not mean random

The term unstructured data can be misleading.

A paragraph of English is highly structured.

It contains:

words,
sentences,
grammar,
punctuation,
meaning.

An image contains spatial structure.

Audio contains temporal structure.

Video contains both spatial and temporal structure.

So "unstructured" does not mean:

without structure.

It means:

The information has not been organised into a predefined machine-readable schema for the task we want to perform.

Semi-structured data

Between structured and unstructured data lies another useful category:

semi-structured data.

Consider an email.

It may contain structured fields:

From:
To:
Date:
Subject:

and then an unstructured body of text.

Or consider a web page.

It contains markup identifying:

headings,
paragraphs,
links,
images.

But the actual meaning of the content is much less rigidly defined.

Other examples include:

JSON,
XML,
event logs,
HTML,
metadata attached to media files.

These contain some explicit structure without fitting neatly into a traditional table.

The same event can produce several forms of data

Imagine a customer calls a support centre.

The interaction might produce:

Structured data

Customer ID: 1942
Call start: 14:02
Call duration: 8 minutes
Agent ID: 37

Unstructured data

An audio recording of the conversation.

Semi-structured data

A transcript containing:

timestamps,
speaker labels,
free-form speech.

These all describe the same event.

But they preserve different information.

Structure makes some questions easier

Suppose we want to know:

How long was the average customer-support call?

Structured call-duration data makes this easy.

Now suppose we want to know:

What problems were customers most frustrated about?

The call duration tells us almost nothing.

The audio recording or transcript may contain the answer.

Different representations support different questions.

This is a recurring theme:

The usefulness of data depends on the decision or question we are trying to support.

Turning unstructured data into structured data

Humans have long converted unstructured information into structured fields.

Imagine reading customer emails and recording:

Email	Topic	Sentiment	Resolution
A	Delivery	Negative	Refund
B	Billing	Neutral	Explanation
C	Product	Positive	None

A person has extracted structure from free-form text.

This process can involve:

classification,
information extraction,
annotation,
coding.

It can be extremely useful.

But it also changes the representation.

Extraction is another form of abstraction

Return to the email:

Hi, the delivery arrived this morning but two of the items were damaged. Could you send replacements before Friday?

We might convert it into:

topic = delivery
damaged_items = 2
resolution = replacement
urgency = high

This representation is much easier to analyse.

But notice what has happened.

The original message contained:

wording,
tone,
context,
sequence,
nuance.

Our structured representation contains only the pieces we chose to extract.

So:

UNSTRUCTURED INFORMATION

↓

EXTRACTION

↓

STRUCTURED VARIABLES

This is another form of abstraction.

Structure can destroy information

Suppose we classify customer reviews as:

positive,
neutral,
negative.

Consider:

The product is excellent, but delivery took three weeks and customer service never responded.

What is the correct label?

Perhaps:

positive

because the product was praised.

Perhaps:

negative

because the service was poor.

Perhaps:

mixed

The original text contains several dimensions.

Reducing it to one category necessarily loses some information.

This does not mean classification is useless.

It means we should understand what the transformation does.

The schema determines what can be recorded

Imagine a hospital database with fields for:

patient age,
diagnosis,
treatment,
discharge date.

Suppose there is no field for:

difficulty accessing transport to the hospital

That information might be extremely important to patient outcomes.

But unless someone records it elsewhere, it does not appear in the structured dataset.

The schema therefore acts as a filter.

It defines which aspects of reality the information system expects to exist.

What does not fit may disappear

Structured systems are powerful partly because they require consistency.

But reality does not always fit predefined categories.

Imagine a form asking:

Reason for appointment:

illness,
injury,
routine check-up,
other.

Someone whose situation does not fit the first three categories becomes:

other.

The information system has simplified the person's experience to fit its schema.

This happens constantly in administrative data.

"Other" is still a modelling decision

The category:

other

can hide enormous variation.

It might contain:

rare conditions,
unusual circumstances,
incorrectly classified cases,
new phenomena the schema was never designed to represent.

If a machine-learning model later learns from this dataset, it inherits those categories.

The model sees the structure created by the information system.

It does not see the original reality that existed before categorisation.

Structured data can still be messy

Structured does not mean clean.

A table may contain:

missing values,
incorrect values,
inconsistent units,
duplicated records,
outdated information,
typing errors.

For example:

Temperature
18.2
19.1
182
17.9

Perhaps:

182

should have been:

18.2

The data is perfectly structured.

It is still wrong.

Structure tells us about organisation, not truth.

Unstructured data can contain rich information

A doctor's clinical note may be much harder for a computer to process than a database field.

But it may contain information such as:

symptoms,
uncertainty,
patient concerns,
context,
reasoning,
observations.

A structured field such as:

diagnosis = asthma

is easier to analyse.

But the clinical note may explain:

why the diagnosis was made,
which alternatives were considered,
how confident the clinician was.

Unstructured data can therefore preserve richness that structured data removes.

This creates a trade-off

Structured data offers:

consistency,
efficiency,
easy querying,
easy aggregation.

Unstructured data offers:

richness,
context,
flexibility,
nuance.

The trade-off can be thought of as:

STRUCTURE → easier computation

while:

LESS STRUCTURE → potentially richer information

Modern AI has become particularly important because it has made the second category much easier to process.

Traditional computing preferred structure

For much of computing history, machines worked most effectively with explicitly structured information.

If we wanted a computer to process something, we often first had to convert it into:

fields,
categories,
rules,
tables.

For example, a traditional customer-support system might require a human agent to choose:

Complaint type: Delivery
Severity: High
Resolution: Replacement

The computer could then operate on those fields.

But the human had already performed the difficult task of interpreting the conversation.

Machine learning began changing this

Machine-learning systems became increasingly capable of learning directly from less structured inputs.

Instead of manually extracting variables from an image, a computer-vision model could learn useful representations from pixels.

Instead of manually coding every property of speech, a speech model could learn from audio.

Instead of manually categorising every document, a language model could learn patterns directly from text.

This shifted some of the work from:

human-designed representation

towards:

machine-learned representation.

Deep learning accelerated the shift

Deep learning made this particularly powerful.

Traditional image recognition might involve humans designing features for:

edges,
shapes,
textures.

A deep neural network can learn many of those representations automatically.

Likewise:

RAW AUDIO → LEARNED REPRESENTATION

and:

RAW TEXT → LEARNED REPRESENTATION

This allowed machines to work much more effectively with information that had previously been difficult to structure manually.

Modern AI can create structure from unstructured information

Consider an AI system reading an invoice.

The invoice may be a PDF or photograph.

The system can extract:

supplier = Example Ltd
invoice_number = 48321
date = 2026-08-06
total = €1,842.50

The input may be relatively unstructured.

The output is structured.

So:

DOCUMENT

↓

AI MODEL

↓

STRUCTURED FIELDS

This is increasingly common in modern information systems.

Language models make this especially powerful

Large language models can often transform free-form language into structured representations.

For example:

Input:

John would like the meeting moved from Tuesday afternoon to Thursday morning because he will be travelling.

Output:

requested_action = reschedule
original_time = Tuesday afternoon
preferred_time = Thursday morning
reason = travel

This ability allows natural language to become an interface to structured systems.

Structure can also be turned back into language

The transformation can work in the opposite direction.

Suppose a database contains:

account_balance = €2,400
payment_due = €180
due_date = 18 August

An AI system might generate:

Your current balance is €2,400. A payment of €180 is due on 18 August.

So we can move between:

STRUCTURED DATA → LANGUAGE

and:

LANGUAGE → STRUCTURED DATA

Modern AI increasingly acts as a bridge between these representations.

Search changes too

Traditional database search might ask:

region = "Dublin"
price < 500000
bedrooms >= 3

This works well when the data is structured.

But suppose we have thousands of reports.

We might want to ask:

Which documents discuss concerns about electricity network congestion caused by new industrial demand?

The relevant concept may appear using many different words.

Traditional exact keyword matching may struggle.

Modern language representations can instead search according to semantic similarity.

Embeddings create searchable structure

Recall embeddings from the previous lesson.

A document can be transformed into a numerical vector representing aspects of its meaning.

So:

DOCUMENT → EMBEDDING

A user's question can also become:

QUESTION → EMBEDDING

The system can then compare the two representations.

Documents whose meanings are similar to the question may appear close together in the embedding space.

This makes large collections of unstructured information much easier to search.

Structured and unstructured data increasingly work together

Modern systems rarely need to choose only one.

Imagine a hospital AI system.

Structured inputs might include:

age,
blood pressure,
medication,
laboratory values.

Unstructured inputs might include:

clinical notes,
scans,
recorded consultations.

The model may combine all of them.

Likewise, an autonomous vehicle might use:

numerical speed measurements,
GPS coordinates,
camera images,
radar signals,
maps.

Intelligence often requires combining many representations of the same world.

Metadata connects the two

Unstructured information often comes with structured metadata.

A photograph might contain:

timestamp,
camera type,
GPS location,
image resolution.

An audio recording might contain:

duration,
recording date,
speaker ID.

A document might contain:

author,
creation date,
title,
file type.

The content may be unstructured.

The metadata is structured.

Both can be useful.

Structure depends on the task

Consider a photograph.

For a traditional database, the image itself may be unstructured.

But a computer-vision system sees a highly organised grid of pixels.

Likewise, language looks unstructured from the perspective of a relational database.

But a language model sees an ordered sequence of tokens.

So the distinction between structured and unstructured data is partly relative to:

the computational system

and:

the task.

What appears unstructured to one system may contain highly exploitable structure for another.

Data can move through many representations

Consider a customer phone call.

It might move through the following pipeline:

SPEECH

↓

AUDIO WAVEFORM

↓

TRANSCRIPT

↓

TOKENS

↓

EMBEDDING

↓

EXTRACTED INFORMATION

↓

DATABASE FIELDS

The same underlying interaction has now been represented in many different ways.

Each representation makes some operations easier.

Each may also lose or transform information.

Representation is a design decision

Suppose we have one million customer-support calls.

We could store:

only the final complaint category

or:

the complete audio

or:

a transcript

or:

a transcript plus structured fields

These choices affect:

storage requirements,
privacy,
cost,
searchability,
model performance,
future possibilities.

Data architecture is therefore not simply a technical afterthought.

It determines what future analysis is possible.

Unstructured data creates resource demands

Working with rich data can be computationally expensive.

A table containing:

customer ID + age + balance

is relatively compact.

A collection containing:

millions of images,
thousands of hours of video,
billions of documents

can require enormous amounts of:

storage,
memory,
network bandwidth,
computation.

This begins to connect data directly to the physical infrastructure behind AI.

Rich representations require resources.

More data is not automatically better

Suppose we have:

one billion documents

That sounds impressive.

But perhaps many are:

duplicated,
low quality,
incorrect,
irrelevant,
badly formatted.

The quantity of data does not guarantee its usefulness.

A smaller, carefully selected structured dataset may outperform an enormous collection of poorly selected unstructured information for a particular task.

The important question is not simply:

How much data do we have?

but:

What information does the data contain, and how useful is it for the task?

Structure can encode assumptions

Suppose a hiring database contains:

applicant
interview_score
hired

This looks objective.

But someone decided:

what the interview score means,
how it is calculated,
which applicants received interviews,
what counts as "hired".

The structured dataset therefore contains assumptions inherited from the process that created it.

A neat table can make those assumptions easy to overlook.

Messiness can contain information

Sometimes irregularity itself is informative.

Suppose medical records contain unusually long free-text notes for particularly complicated patients.

If we discard the text and retain only structured diagnosis codes, we may lose evidence of complexity.

Likewise, repeated customer messages may indicate frustration.

The messiness of the data can itself reflect something about the underlying system.

Cleaning and structuring data should therefore be done carefully.

AI does not eliminate the need for structure

Because modern AI can process unstructured information, it may be tempting to conclude that schemas and databases no longer matter.

They do.

Structured data remains extremely valuable for:

precise transactions,
accounting,
measurement,
auditing,
operational systems,
reliable querying.

An AI system might interpret:

Transfer €50 to Alex.

But the actual financial system still needs precise structured information such as:

source_account
destination_account
amount
currency
timestamp

Natural language can help humans interact with the system.

The underlying operation still requires structure.

Intelligence often sits between representations

This suggests an important role for AI.

AI can act as an interface between:

rich human information

and:

precise machine systems.

For example:

HUMAN LANGUAGE

↓

AI INTERPRETATION

↓

STRUCTURED REQUEST

↓

COMPUTER SYSTEM

and then:

STRUCTURED RESULT

↓

AI INTERPRETATION

↓

HUMAN LANGUAGE

This is one reason language models can become powerful interfaces to software and services.

Structure and uncertainty

There is another important issue.

Suppose an AI extracts:

complaint_type = delivery

from an email.

How certain is it?

Perhaps:

delivery = 0.82
billing = 0.12
product = 0.06

The structured output can make the answer appear definite even when the underlying interpretation was uncertain.

When AI converts ambiguous information into clean categories, uncertainty can disappear from view.

This is dangerous.

A precise-looking field can originate from an uncertain inference.

We will return to this throughout the course.

Human language is inherently ambiguous

Consider:

I'll meet you by the bank.

Which bank?

A financial institution?

A river bank?

Perhaps the previous conversation makes it obvious.

Perhaps it does not.

When unstructured information is converted into structured fields, systems may have to resolve ambiguity.

Sometimes the correct response should not be:

choose one

but:

I am uncertain.

This connects directly to our later module on probability and inference.

Structured outputs can create an illusion of certainty

Suppose an AI reads a medical note and produces:

diagnosis = pneumonia

A database field looks authoritative.

But perhaps the original note said:

Possible pneumonia, although viral infection cannot yet be ruled out.

The transformation has removed uncertainty.

A better representation might preserve it:

pneumonia_probability = 0.60
alternative = viral infection
confidence = moderate

How we structure information can therefore affect how uncertainty is communicated.

From data to decisions

The distinction between structured and unstructured information becomes particularly important once data feeds automated decisions.

Consider:

EMAIL

↓

AI INTERPRETATION

↓

PRIORITY = HIGH

↓

QUEUE POSITION

The model has transformed unstructured language into a structured variable.

That variable then affects resource allocation.

The complete chain is:

UNSTRUCTURED REALITY

↓

REPRESENTATION

↓

MODEL

↓

STRUCTURED PREDICTION

↓

DECISION

↓

ACTION

This is where apparently technical decisions about data representation begin to have real consequences.

Structure can become institutional reality

Imagine an organisation creates three customer categories:

low priority,
standard priority,
high priority.

Once these categories exist, the organisation may begin designing:

queues,
staffing,
pricing,
service levels

around them.

The categories begin as a representation.

But they can eventually shape the real service people receive.

So:

REALITY → DATA CATEGORIES → DECISIONS → NEW REALITY

The representation can feed back into the world.

AI dramatically expands what can become data

Historically, much human activity was difficult for computers to process.

A conversation might simply disappear after it happened.

A photograph might be stored but not easily searchable by meaning.

A handwritten document might require a person to interpret it.

Modern AI increasingly allows machines to extract useful representations from:

speech,
images,
video,
handwriting,
natural language.

This dramatically expands the amount of human activity that can be made computationally accessible.

That creates enormous opportunities.

It also creates serious questions about:

privacy,
surveillance,
ownership,
consent,
power.

We will return to these questions later.

The boundary is becoming less important

The historical distinction was roughly:

structured data → easy for computers

unstructured data → difficult for computers

Modern AI is weakening that distinction.

Language models can process documents.

Vision models can process images.

Speech models can process conversations.

Multimodal systems can combine them.

So increasingly:

UNSTRUCTURED INFORMATION → LEARNED REPRESENTATION → COMPUTABLE INFORMATION

This is one of the most important transformations enabled by modern AI.

But representation remains unavoidable

AI does not somehow access meaning directly.

An image still becomes numerical values.

Speech still becomes numerical signals.

Text still becomes tokens.

Those inputs become internal representations.

The representation may now be learned rather than manually designed.

But the fundamental pipeline remains:

REALITY

↓

MEASUREMENT

↓

DATA

↓

REPRESENTATION

↓

MODEL

The machine still works with a representation of the world.

Ask how the information is structured

Whenever you encounter a dataset, ask:

Is the information structured, semi-structured or unstructured?
What schema exists?
Who designed that schema?
What information fits easily into it?
What information does not?
What was lost when the data was categorised?
Is rich context available elsewhere?
Can AI extract useful structure from the original information?
How certain is that extraction?
Could the structured output make uncertain information appear definite?
Does the representation preserve what matters for the eventual decision?

These questions help reveal the assumptions hidden inside information systems.

The central idea

Structured data makes information explicit.

It tells the computer:

what each value represents

and:

where it belongs.

Unstructured data preserves information in richer forms such as:

language,
images,
audio,
video.

Historically, computers strongly preferred the first.

Modern AI has dramatically improved our ability to work with the second.

But the deeper principle has not changed:

Information must always be represented somehow, and every representation preserves some things while discarding others.

A neat database table is not necessarily closer to truth than a messy document.

It is simply easier to compute with.

And when unstructured information is transformed into structured variables, we should remember that the transformation itself may involve:

interpretation,
uncertainty,
abstraction,
judgement.

In the next lesson, we will examine what happens when information is not available at all:

missing data — why observations disappear, why the pattern of what is missing can itself contain information, and why simply deleting incomplete records can seriously distort what a model learns.