Contextual Intelligence Bo Begole, Ph.D.

Samsung Research America, UX Center bo@begole.net

Korean HCI Society Conference 31 January 2013

Businesses strategies for technologies that bridge physical and digital worlds

•  Capabilities, Techniques,

Limitations •  How to exploit the trends •  Key value propositions •  Case Studies in

•  Contextual Intelligence •  Hyper-Personal information

filtering •  Predictive personal marketing •  Supply chain disruption •  Unanticipated device

interoperation •  Consumer decision support

FT Press, ISBN-13: 978-0-13-706443-4

20 Years ago, Ubiquitous Computing was a dream …

3

Olivetti Active Badge

PARC Tab PARC Pad PARC “Deathstar” IR Network Hub

“The most profound technologies are those that disappear.

They weave themselves into the fabric of everyday life until they are

indistinguishable from it.” -Mark Weiser, Xerox PARC

http://www.youtube.com/watch?v=b1w9_cob_zw

Too Many Devices!

Too Much Information!

Too Many Friends!

Too Many Networks!

Too Much To Do!

Context-Aware services reduce technology overload

Systems detec users’ locations to help them

–  Find colleagues –  Find resources –  Seamlessly connect to

nearby devices/services –  Route phone calls –  Tag information creation

by context for recall –  Diary of daily activities (1994) Bill Schillit, Norm Adams, Roy Want

Context-Aware Computing Applications Workshop on Mobile Computing Systems and Applications,

6

7

1990 92 96 2000 02 2004 94 98

Commercial penetration of Ubiquitous Computing research is accelerating

Context-Aware research

PARC Magitti

8

Context-Aware research

1990 10 92 96 2000 02 04 06 08 94 98

Google Now

Mind- meld

Commercial penetration of Ubiquitous Computing research is accelerating

12

CALO ends

Siri launches

Context-aware is now table-stakes. New frontier: Behavior, Activity and Goal-Aware

Context-Aware Sensor-triggered services with no intelligence: Current/future location; time

Key: distance in time/space 9

Incr

easi

ng In

telli

genc

e

1990-2004

Predictive analytics from behavior traces: rank info by frequency/recency of people, apps, docs, locations; …

Key: distance from “normal”

Behavior & Activity-Aware

2005-2015

Semantic understanding of workflows and objectives: writing a proposal; filling sales pipeline; shopping for gift; fostering relationships; …

Key: distance from “objective”

Goal-Aware

2015+

Sternberg’s Tri-archic Theory of Intelligence

• Analytic Intelligence – Logic, sequences, calculations, …

• Synthetic Intelligence – Creativity, intuition, …

• Contextual Intelligence – Practical “street smarts” – Understanding relationships and utility – Achieve goals with available resources

10

Contextually Intelligence Assistant: Pertinent and Actionable information

11 http://youtu.be/FPL7uaKYNiU

Nice vision, but it’s been 20 years…

• Some classic problems are still unsolved… – Privacy – Missing information

• … and bigger problems have arisen – Latencies

•  Delays to build model •  Delays in detecting pattern change •  Delays to detect situation

– Accuracy: typically 85% at best – Actionable? Tell me something I don’t know

12

Privacy Concerns

• Security and access control is only half the story

•  Impression Management –  Confusing interfaces,

contradicting policies –  What is this thing telling other

people about me? • Economic Risk

–  Unwanted sales pressure –  Higher insurance rates

13

Current digital assistants have to ask for every detail

14

“The address was there in my contact card, but Siri kept telling me, ‘I don’t know where your

home is.’”

Why do I have to tell it everything? Why can’t it figure things out and confirm?

Latencies

• Pattern models take time to build –  Frequent patterns build quickly –  Weekly, monthly, annual patterns build slowly

• Changes take time to detect

• Recognition is often slower than human intelligence

15

GPS

NFC

EMAIL

Calendar

Accuracy – best are wrong 15-20%

16

• Fine for advertising or product recommendations • What is good enough for content retrieval?

Optimal Timing

Action Context

Reminders in actionable context

Pertinent Information & Actionable Context n  Communication: tell me

before someone becomes unavailable

n  Health & Fitness: suggestions of healthy options not already in practice

n  Technician: take preventive parts before repair visit

n  Advertisement: Deliver ads when person is actually “in the market”

•  Health plan •  Task list •  Reminders •  Coupons

Technology alone cannot solve these •  Privacy – Requires understanding real user fears •  Missing information – Human conversational repair •  Latencies – Give users control of their pattern models •  Accuracies – Adjust UX depending on application need •  Actionability – Don’t be a nag. What do they already know?

What will achieve their goal? How is their goal defined?

HCI Grand Challenge:

HCI research methods can create the knowledge to emulate human contextual intelligence

18