IBM IOD Conference 2011 Opening Keynote Deck

© 2011 IBM Corporation

Big Data. Deep Analytics. New Physics.The Journey from Enterprise Amnesia

to Enterprise Intelligence

Jeff Jonas, IBM Distinguished EngineerChief Scientist, IBM Entity Analytics

Email: [email protected]: www.jeffjonas.typepad.com

Twitter: http://www.twitter.com/jeffjonas


State of the Union:Enterprise Amnesia


Amnesia, definition

A defect in memory, especially resulting from brain damage.


Enterprise Amnesia, definition

A defect in memory, resulting in missed opportunity, wasted resources, lower revenues, unnecessary fraud losses, and other bad news.


Time

Trend: Organizations Are Getting Dumber

Sensemaking Algorithms

Available Observation

Space

Structured data Unstructured data Social media Cyber audit logs Geospatial data

Com

pu

tin

g P

ow

er

Gro

wth

EnterpriseAmnesia


Data Volumes Exploding

“Every two days now we create as much information as we did from the dawn of civilization up until 2003.”

~ Eric Schmidt, CEO Google


WHY?

Trend: Organizations Are Getting Dumber

Time



Space

Com

pu

tin

g P

ow

er

Gro

wth


Algorithms at Dead End.

You Can’t Squeeze Knowledge

Out of a Pixel.


No Context

[email protected]


Context, definition

Better understanding something by taking into account the things around it.


Information in Context … and Accumulating

Top 200Customer

Job Applicant

IdentityThief

CriminalInvestigation

[email protected]


The Puzzle Metaphor

Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes and colors

What it represents is unknown – there is no picture on hand

Is it one puzzle, 15 puzzles, or 1,500 different puzzles?

Some pieces are duplicates, missing, incomplete, low quality, or have been misinterpreted

Some pieces may even be professionally fabricated lies

Until you take the pieces to the table and attempt assembly, you don’t know what you are dealing with


Puzzling

270 pieces90%

200 pieces66%

150 pieces50%

6 pieces2%(pure noise)

30 pieces10% (duplicates)


First Discovery


More Data Finds Data


Duplicates in Front Of Your Eyes


First Duplicate Found Here


Incremental Context – Incremental Discovery

6:40pm START

22min “Hey, this one is a duplicate!”

35min “I think some pieces are missing.”

37min “Looks like a bunch of hillbillies on a porch.”

44min “Hillbillies, playing guitars, sitting on a porch, near a barber sign … and a banjo!”


150 pieces50%


Incremental Context – Incremental Discovery

47min “We should take the sky and grass off the table.”

2hr “Let’s switch sides, and see if we can make sense

of this from different perspectives.”

2hr10m “Wait, there are three … no, four puzzles.”

2hr17m “We need a bigger table.”

2hr18m “I think you threw in a few random pieces.”


How Context Accumulates

With each new observation … one of three assertions are made: 1) Un-associated; 2) placed near like neighbors; or 3) connected

Must favor the false negative

New observations sometimes reverse earlier assertions

Some observations produce new discovery

As the working space expands, computational effort increases

Given sufficient observations, there can come a tipping point. Thereafter, confidence improves while computational effort decreases!


Big Data [in context]. New Physics.

More data: better the predictions– Lower false positives

– Lower false negatives

More data: bad data … good– Suddenly glad your data was not perfect

More data: less compute


Enterprise IntelligenceOne Plausible Journey

Enterprise IntelligenceHow to Get From Here to There


ObservationSpace

Sense and Respond

What you know

New Observations


ObservationSpace

Decide

?Relevance

Finds the Sensor(<200ms)

Data Finds Data

Sense and Respond


Explore and Reflect

ObservationSpace

Decide

?DirectedAttention

Relevance Find You

DeepReflection

CuratedData

PatternDiscovery

RelevanceFinds the Sensor

(<200ms)

Data Finds Data

Sense and Respond


ObservationSpace

Decide

?DirectedAttentionNEW

INTERESTS

DeepReflection

CuratedData

PatternDiscovery


(<200ms)

Data Finds Data

Explore and ReflectSense and Respond


ObservationSpace

Decide

?

DeepReflection

CuratedData

PatternDiscovery


(<200ms)

Data Finds Data

ILog

NetezzaBigInsights

DirectedAttention

Cognos


InfoSphere Streams

NEWINTERESTS


ObservationSpace

Decide


INTERESTS

DeepReflection

CuratedData

PatternDiscovery


(<200ms)

Data Finds Data

Report and Manage



Decide


INTERESTS

CuratedData

PatternDiscovery


(<200ms)

Data Finds Data

Info Management Systems

Content ManagementCase Management Data Warehousing

Report and Manage


ObservationSpace

Decide

?

DeepReflection

CuratedData

PatternDiscovery


(<200ms)

Data Finds DataIdentity Insight/Sensemaking

SPSS

ILog

NetezzaBigInsights

DirectedAttention

Cognos


InfoSphere Streams

NEWINTERESTS


Closing Thoughts


The most competitive organizations

are going to make sense of what they are observing

fast enough to do something about it

while they are observing it.


Wish This On The Competition

Time



Space

Com

pu

tin

g P

ow

er

Gro

wth

EnterpriseAmnesia


The Way Forward: Enterprise Intelligence

Time



Space

Com

pu

tin

g P

ow

er

Gro

wth

Context Accumulation


Big Data. Deep Analytics. New Physics.The Journey from Enterprise Amnesia

to Enterprise Intelligence

Jeff Jonas, IBM Distinguished EngineerChief Scientist, IBM Entity Analytics

Email: [email protected]: www.jeffjonas.typepad.com

Twitter: http://www.twitter.com/jeffjonas

IBM IOD Conference 2011 Opening Keynote Deck

Technology

Transcript of IBM IOD Conference 2011 Opening Keynote Deck