BIG DATA COMES IN SMALL PACKAGESHow the Internet of Things Will Change Big Data’s Future

Daniel AustinGRIN Technologies, Inc.The Connected Life Companydaniel.austin@grintech.net

NoSQL Now! August 21st, 2014

V1.1

ABOUT GRIN TECHNOLOGIES, INC.

• Founded July 4th 2014• “The Connected Life Company”• Developing Systemsware at the Intersection of Big Data and the Internet of

Things• Smart People Working Hard to Solve Real Problems• Coming in Q3 2015: The Connected Life Server ™

• A Platform for Connecting People, Places, and Things

Soft launching today at NoSQL Now!

FOUR BIG IDEAS FOR TODAY’S TALK

1. No “Internet of Things”, only disjoint Networks of Things2. The IoT is an “iceberg” Big Data problem3. Things are dumb, systems are smart4. The “Witnesses” principle

WHY THERE WILL BE NO “INTERNET OF THINGS”

• No clients; no servers• No network that connects them all• No protocols that make sense

• Intermittent, Transient, Low Value, Connectionless

Many of Our Current Assumptions About the Internet Simply Don’t Apply!

INSTEAD, DISJOINT NETWORKS OF THINGS

• Disjoint – for Security and Privacy• Functional and Context-Specific• Mesh-based Peering Architectures

Peer 1

Peer 2

Peer3

Peer 4

Peer 1

Peer 2

Peer3

Peer 4Public

Internet

Gateway Gateway

LAYER 8 HYPOTHESIS

• You are a Node on the Internet!• The Internet Connects People, Not Computers• Humans are Machines that Turn Data into Information

THE CONVERGENCE OF THE IOT AND BIG DATA

It’s not an accident that these technologies are coming together at the same time; the IoT wasn’t possible before we had solutions to Big Data problems.

Things

Big Data

Big Metadata

METADATA EXPLOSION

• “Big Metadata” is a bigger problem than Big Data

• Austin’s Law: “The Amount of Data to be Crunched Will Always Exceed the Computing Power Available.”

• What are these companies’ data retention policies?

ARE YOU A BIG DATA PROBLEM?

• Each person will generate roughly 25 petabytes of data over the course of a lifetime• Users have little control over collection and storage• Email, documents, receipts, bills (!), your car, music, books…

• Pervasive computing multiplies the problem• Majority of data valueless out of context • Security & privacy risks may outweigh the benefits ?!

LET’S TALK LIGHT BULBS

• Invented in 1879 by Edison and Swan• Many others before had failed• One of humanity’s most important

inventions• More than 400 distinct types are

manufactured• Streets, Airplanes, Neon (Still 82%

Incandescent!)• The Eiffel Tower uses 20,376 light bulbs

• Many Lighting Systems Are Extremely Complex! • Not a simple on/off state machine• Critical for many life-and-death situations

• Unsustainable and unmanageable at scale• Arrogant, disposable “dumb” design

• 2013: First light bulb on the Internet

IMAGINING THE IOLB

What If…Every light bulb was connected to the IoT?

What Would the Architecture Look Like?Could We Manage the Data Usefully?Could We Build it?

LIGHT BULBS ARE DUMB; SYSTEMS ARE SMART

• We can’t design the IoLB by only changing the light bulbs; it’s a system

• It’s necessary for the system to evolve, not just the bulbs• Bulbs will change to meet new conditions, become smarter

• Sockets, power lines, power generators and coal mines• Every part of the system will evolve at a different rate

• While building the IoT, we will need to consider how to build smart systems that help ordinary users, requiring compromise (and patience)

SYSTEMS THINKING AND LIGHT BULB ARCHITECTURE

WHICH COMES FIRST? THE SOCKET OR THE BULB?

• Problem: Do we want smart light bulbs or smart sockets?• Light bulbs are easy to change, sockets are not• But in an ideal world, most of the architectural ‘smarts’ of the

IoLB would be in the socket

• This is a classic architectural trade-off• It’s too expensive to upgrade every part of the system at the same time!• Our light bulbs will be smarter than their sockets• For some time; this will have wider implications• For the IoLB (and IoT)

THE IOLB AND BIG DATA

12x10^9 light bulbs x (365 state changes/day) x 2 x 22 bytes = Big DataA more realistic estimate would be 3 orders of magnitude larger! Plus the history, maintenance, etc.

SECURITY AND PRIVACYON THE IOT

• Security Is Protection from Harm• Privacy is Filtering Data Based on Context

• The Separation of the Self from the Group• Persona v. Identity

THE “WITNESSES” PRINCIPLE

• Cellphones are “things”, so are security cameras…• “When There’s a Witness to Every Act…”

By 2020 everyone, everything and everywhere will be connected in real time. More than 50 billion connected devices will be used in the Networked Society.Source: http://www.ericsson.com/thinkingahead/networked_society

IN A WORLD OFTEN BILLION PEOPLE AND THEIR THINGS…

• Everyone will be connected – to each other• Privacy will disappear• Security (and civility) will increase• Witnesses are a good thing ™

A SPECIES AND ITS DATA: THE LONG VIEW

• We can’t build to scale exponentially. Period.

• We are all part of the network• The purpose of evolution is

computation

Will humanity merge with it’s data?

The Age of Big Metadata

The Age of Big Data

THANKS!“When everyone’s life is connected, that’s when the Singularity will happen.”

Daniel AustinGRIN Technologies, Inc.The Connected Life Companydaniel.austin@grintech.net

http://www.grintech.net@daniel_b_Austin

@GRINTechInc