The Joy of Data

- It sometimes seems we're being deluged with data.

Wave upon wave of news and messages.

Submerged by step counts.

Constantly bailing out to make room for more.

We buy it, surf it, occasionally drown in it

and with modern technology quantify

ourselves and everything else with it.

Data is the new currency of our time.

- Data has become almost a magic word for anything.

Crime and lunacy and literacy

and religion and drunkenness.

You name it, somebody was gathering information about it.

- It offers the ability to be transformationally positive.

- It's, in one sense, just the reduction in uncertainty.

So what exactly is data?

How is it captured, stored, shared and made sense of?

The engineers of the data age are

people that most of us have never heard of,

despite the fact that they brought about

a technological and philosophical revolution,

and created a digital world that

the mind boggles to comprehend.

This is the story of the word of our times.

How the constant flow of more

and better data has transformed society.

And is even changing our sense of ourselves.

I can't believe this is my life now.

So come on in, because the water's lovely.

My name is Hannah Fry, I'm a mathematician,

and I'd like to begin with a confession.

I haven't always loved data.

The truth is mathematicians just

don't really like data that much.

And for most of my professional life I was quite happy

sitting in a windowless room with my equations,

describing the world around me.

You can capture the arc of a perfect free kick

or the beautiful aerodynamics of a race car.

The mathematics of the real world is

clean and ordered and elegant,

everything that data absolutely isn't.

There was one moment that helped to change my mind.

It was in 2011 when I came across a little game that

a teenage Wikipedia user called Mark J had invented.

Now, Mark noticed that if you hit the first

link in the main text of any Wikipedia page

and then do the same for the next page, a pattern emerges.

So the page for data, for example,

links from "set" to "maths"

to "quantity" to "property" and then "philosophy",

which after a few more links will loop back onto itself.

Now, the page "egg" ends up in the same place,

and even that famously philosophical boyband

One Direction will take you

all the way through to "philosophy",

although you have to go through "science" to get there.

The same goes for "fungi", or "hairspray",

"marmalade", even "mice", "dust" and "socks".

It was a very strange finding

and it called for some statistics.

Another Wikipedia user, Il Mare,

wrote a computer program to try

and investigate this phenomenon.

Now, he discovered, amazingly,

that for almost 95% of Wikipedia pages,

you will end up getting to "philosophy" eventually.

Now, that's pretty cool,

but how did it change my mind about data?

Well, the pattern that Mark J discovered

and the data that was captured and analyzed,

it revealed a hidden mathematical structure,

because Wikipedia is just a network with

loops and chains hidden all over the place

and it's something that can be

described beautifully using mathematics.

For me this was the perfect example of

how there are two parallel universes.

There's the tangible, noisy, messy one,

the one that you can see and touch and experience.

But there's also the mathematical one,

where I think the key to our understanding lies.

And data is the bridge between those two universes.

Our understanding of everything from cities to crime,

global trade, migration and even disease,

it's all underpinned by data.

Take this for example.

Rural Wiltshire and a dairy farm

gathering data from its cows wearing pedometers.

We can't be out here 24-7.

The pedometers help us to have our eyes and ears everywhere.

- It turns out when cows go into heat

they move around a lot more than normal.

Constant monitoring of their steps and some background

mathematics reveal the prime time for insemination.

- We'll be able to look at the data

and within 24 hours there'll be a

greater chance of getting her in calf.

- Data-driven farming is now big business,

turning a centuries-old way of life into precision science.

Pretty much every industry you can

think of now relies on data.

We all agree that we are undergoing

a major revolution in human history.

The digital world replacing the analog world.

A world based on data,

they are made of codes rather than

a world made of biological or physical data,

that is extraordinary.

Why philosophy at this stage?

Because when you face extraordinary challenges,

the worst thing you can do is to get close to it.

You need to take a long run-up.

The bigger the gap, the longer the run-up.

And the run-up is called philosophy.

- In the spirit of taking a long run-up,

we'll start with the word itself.

Data is originally from the Latin datum,

meaning "that which is given".

Data can be descriptions.

Counts, or measures.

Of anything.

In any format.

It's anything that when analyzed becomes information,

which in turn is the raw material for knowledge,

the only true path to wisdom.

Look at the data on data.

And before the scientific and industrial revolution,

the word barely gets a look in, in English.

But then it starts to appear in print

as scientists and the state gather,

observe and create more and more of it.

This arrival of the age of data would change everything.

Industrial Revolution Britain.

For Victorians, booming industry

and the growth of major cities were changing both the

landscape and daily life beyond recognition.

Into this scene stepped an unlikely man of numbers,

William Farr, one of the first people

to manage data on an industrial scale.

- William Farr had a quite unusual upbringing,

in that he was actually the son of a farm laborer

but who had managed to get a medical education,

which was really very unusual for someone of his class.

Farr very quickly became absorbed

in the study of statistics.

He was particularly interested,

as you might expect for somebody with medical training,

in public health, life expectancy and about causes of death.

For anyone interested in statistics,

there was only one place to be.

Somerset House in London was home to

the General Register Office,

where, in 1839, Farr found his dream job.

From up there in the north wing,

William Farr, the apothecary,

medical journalist and top statistician,

would really rule the roost.

Now, this place was almost like a factory.

Here, they would collect, process

and analyze vast amounts of data.

So in would come the census returns,

the records of every single birth,

death and marriage in the country,

and out would come the big picture,

the usable information that could help

inform policy and reform society.

- I think it's sometimes difficult for us to remember

just how little people knew in the early 19th century

about the changes that Britain was going through.

So when Farr did an analysis of

population density and death rate,

he was able to show that life expectancy

in Liverpool was absolutely atrocious.

It was far, far worse than the surrounding areas.

This came as a surprise to a lot of people

who believed that Liverpool, a coastal town,

was actually quite a salubrious place to live.

At Somerset House, Farr spearheaded

a revolution in the systematic collection of data

to uncover the real picture of this changing society.

Its scale and ambition was described

in a newspaper at the time,

"In arched chambers of immense strength and extent are,

"in many volumes, the genuine certificates of

"upwards of 28 million persons born into life,

"married or passed into the grave."

Here, every person was recorded equally.

A revolutionary idea.

"Here are to be found the records of nonentities,

"side-by-side with those once learned in the law

"or distinguished in literature, art or science."

But what really motivated William Farr

was not just data collection,

it was the possibility that data gathered could be

analyzed to help overcome society's greatest ill.

Cholera was probably the most feared

of all of the Victorian diseases.

The terrifying thing was that you could wake up

in the morning and feel absolutely fine,

and then be dead by the evening.