Cognition at Stanford Jay McClelland & Lera Boroditsky Spring, 2009.
AUGMENTED LARGE SCALE COGNITION · 2015. 1. 21. · AUGMENTED LARGE SCALE COGNITION Stuart Card...
Transcript of AUGMENTED LARGE SCALE COGNITION · 2015. 1. 21. · AUGMENTED LARGE SCALE COGNITION Stuart Card...
-
AUGMENTED LARGE SCALE COGNITION
Stuart Card Palo Alto Research Center (PARC) (Visiting Professor, Stanford)
Stanford Computer Forum Stanford University April 15, 2009
-
ENGINEERING HUMAN PROSTHESES
-
HOW DO WE DESIGN COGNITIVE PROSTHESES?
? Meg Stewart
-
Paul MacCready
Motivating Problem: Large-Scale Cognition
7.3 million pages/day
-
Comparisons
1999 2009 (EB) (EB)
Unique Info 0.6 36.0 Store in earth’s population
memory in 1 yr 0.1 0.1 Record all words in all
lives 3.6 3.6
Source: Berkeley SIMS + Computations
-
Man the informavore (George Miller, 1983)
Informavores Hunger for information about
the world Use information to adapt to the
world
-
COGNITION
Find . . . (Perceive, Learn, Remember) Think . . . (Decide) Do . . . (Create, Act)
-
LEVELS OF COGNITION
Immediate behavior Experiential cognition Routine cognitive skill
-
LEVELS OF COGNITION
Problem Solving Reflective cognition
-
LEVELS OF COGNITION
Social cognition People finding, thinking, and doing
together
-
TIME LEVELS OF BEHAVIOR
107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL Automatic Behavior 10-3 (msec) 10-4
-
IMMEDIATE BEHAVIOR
107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL 10-3 (msec) 10-4
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion)
-
MODEL HUMAN PROCESSOR
Processors and Memories applied to human
Used for routine cognitive skill
-
EXAMPLE: ZERO-PARAMETER CALC
Problem: Inventor claims he invented 600 wpm typewriter. License and develop?
Solution 1: Half stroke: τM = 70 ms/char Whole stroke: τM + τM = 140 ms/char
but if between hands, overlap: τM = 70 ms = 131 words/min
-
EXAMPLE: ZERO-PARAMETER CALC
Solution 2: (range calculation) Half stroke: τM=70 [30~100] ms/char = 131 [308~92] words/min
Conclusion: Bogus claim. Throw him
out!
-
TASK ANALYSIS: GOMS (GOALS, OPERATORS, METHODS, SELECTION RULES)
GOAL: EDIT-MANUSCRIPT • repeat until done GOAL: EDIT-UNIT-TASK
GOAL: ACQUIRE-UNIT-TASK • if not remembered GET-NEXT-PAGE • if at end of page GET-NEXT-TASK • if an edit task found
GOAL: EXECUTE-UNIT-TASK GOAL: LOCATE-LINE • if task not on line
[select : USE-QS-METHOD USE-LF-METHOD]
GOAL: MODIFY-TEXT [select USE-S-COMMAND
USE-M-COMMAND]
task analysis
-
PREDICTS TIME WITHIN ABOUT 20%
-
SAE RECOMMENDED PRACTICE J2365
Predict task times for car navigation systems
Check compliance with SAE J2364 (15-Second Rule)
Note: To estimate times while driving, multiply by 1.3 to 1.5.
Based on GOMS and work by Paul Green at Univ. of Michigan Transportation Research Institute.
Dario Salvucci
-
SAE J2365 OPERATOR TIMES Time (s)
Code Name Young (18-30
Old (55-60)
Rn Reach near 0.31 0.53 Rf Reach far 0.45 0.77 C1 Cursor once 0.80 1.36 C2 Cursor 2 times or more 0.40 0.68 L1 Letter or space 1 1.00 1.70 L2 Letter or space 2 times or more 0.50 0.85 N1 Number once 0.90 1.53 N2 Number 2 times or more 0.45 0.77 E Enter 1.20 2.04 F Function keys or shift 1.20 2.04 M Mental 1.50 2.55 S Search 2.30 3.91 Rs Response time of system-scroll 0.00 0.00 Rm Response time of system-new menu 0.50 0.50 Paul Green UMITRI
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output
-
INTERACTIVE COMPUTING
typewriter I/O Graphical CRT
Whirlwind (MIT)
-
DIRECT MANIPULATION
Sketchpad (Sutherland, 1963)
Input on Output
-
J. C. R. LICKLIDER
(Cognitive Prosthetic) HUMAN-MACHINE SYMBIOSIS: “The hope is that in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain ever thought.”
-
THREE THEMES FOR LARGE SCALE COGNITION
Efficient Communication Tight Coupling
Representation Shift
LOOK AT COMBINATORICS OF COGNITIVE LEVELS X THEMES
-
PROBLEM SOLVING
107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL Automatic Behavior 10-3 (msec) 10-4
Meg Stewart
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search)
-
IMMEDIATE BEHAVIOR
Routine cognitive skill Well-known path
-
Information Search
Problem solving Heuristic search Exponential if
don’t know what to do
-
PHASE TRANSITION IN NAVIGATION COSTS AS FUNCTION OF INFORMATION SCENT
Notes: Average branching factor = 10 Depth = 10
0 2 4 6 8 10 0
50
100
150
Depth
Num
ber o
f pag
es v
isite
d
.100
.125
.150
0 2 4 6 8 10 0
50
100
150
.100
.150
Probability of choosing wrong link (f)
0 0.05 0.1 0.15 0.2 0
20
40
60
80
100
f
Num
ber o
f Pag
es V
isite
d pe
r Lev
el
Linear Exponential
-
OPTIMALITY THEORY
Max Useful info
Time Max Energy
Time [ ] [ ]
Optimal Foraging Theory Information Foraging Theory
-
Information Foraging Theory:
patchWithinpatchBetweenWB TTGain
TTGR
−− +=
+=
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search)
-
WITHIN-PATCH ENRICHMENT: INFORMATION SCENT
Tokyo
San Francisco
New York
perception of value and cost of a path to a source based on proximal cues
-
Boosting Information Scent
-
IMPORTANCE FOR WEB DESIGN
Jarad Spool, UIE
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling)
-
RELEVANCE-ENHANCED THUMBNAILS Emphasize text that
is relevant to query Text callouts
Enlarge text that might be helpful in assessing page Enlarge headers
020406080100120140160180
Picture Homepage E-commerce Side-effects
Tota
l Sea
rch
Tim
e (s
)
Text Plain Enhanced
Allison Woodruff
-
MACHINE MODELING OF INFORMATION SCENT
cell
patient
dose
beam
new
medical
treatments
procedures
Information Goal Link Text
-
PREDICTION OF LINK CHOICE
R2 = 0.72
0
5
10
15
20
25
30
35
0 5 10 15 20 25 30 35 Observed frequency
Pre
dict
ed fr
eque
ncy
R2 = 0.90
0
10
20
30
40
50
0 10 20 30 40 50 Observed frequency
Pre
dict
ed fr
eque
ncy
(a) ParcWeb (b) Yahoo
Piroli, PARC
-
BLOODHOUND PROJECT
Starting Point: www.xerox.com Task: look for “high end copiers”
OUTPUT usability metrics
INPUT
Chi, et al
-
Smart Book Semantic Index and Scent Highlighting Aids the analyst in finding the most
relevant information quickly.
Information Scent
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift)
-
VISUALIZATION Re-represent to see patterns (eg.,
amphidromic points).
Ray
-
MACRO-MICRO READING: 6M POINTS
-
VISUALIZATION REFERENCE MODEL
Human InteractionHuman Interaction
VisualVisualMappingsMappings
VisualVisualStructuresStructures
ViewViewTransformationsTransformations
Visual FormVisual Form
ViewsViews
Raw Data: idiosyncratic formatsData Tables: relations (cases by variables) + meta-dataVisual Structures: spatial substrates + marks + graphical propertiesViews: graphical parameters (position, scaling, clipping, …)
RawRawDataData
DataDataTransformationsTransformations
DataData
DataDataTablesTables
TaskTask
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling)
-
INTERACTIVE
Human InteractionHuman Interaction
VisualVisualMappingsMappings
VisualVisualStructuresStructures
ViewViewTransformationsTransformations
Visual FormVisual Form
ViewsViews
Raw Data: idiosyncratic formatsData Tables: relations (cases by variables) + meta-dataVisual Structures: spatial substrates + marks + graphical propertiesViews: graphical parameters (position, scaling, clipping, …)
RawRawDataData
DataDataTransformationsTransformations
DataData
DataDataTablesTables
TaskTask
Dynamic Queries Magic Lens Overview + Detail Linking & Brushing Extraction & Comparison Attribute Explorer
-
DYNAMIC QUERIES
Home Finder (U. Maryland)
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling) Attention-reactive displays (Implicit coupling)
-
ATTENTION REACTIVE
Human InteractionHuman Interaction
VisualVisualMappingsMappings
VisualVisualStructuresStructures
ViewViewTransformationsTransformations
Visual FormVisual Form
ViewsViews
Raw Data: idiosyncratic formatsData Tables: relations (cases by variables) + meta-dataVisual Structures: spatial substrates + marks + graphical propertiesViews: graphical parameters (position, scaling, clipping, …)
RawRawDataData
DataDataTransformationsTransformations
DataData
DataDataTablesTables
TaskTask
-
ATTENTION-REACTIVE
-
Degree-of-Interest Trees
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling) Attention-reactive displays (Implicit coupling) Sensemaking tools (Representation shift)
-
SENSE MAKING TASKS Characteristics
Massive amounts of data Ill-structured task Organization,
interpretation, insight needed
Output, decision, solution required
Examples Understanding a health
problem and making a medical decision
Buying a new laptop Weather forecasting Producing an intelligence
report
-
IMPORTANCE OF SENSE MAKING
75% of “significant tasks” on the Web are more than simple “finding” of information (Morrison et al., 2001) Understanding a topic (e.g., about health) Comparing/choosing products
Information retrieval does not support these tasks (Bhavnani et al., 2002) E.g., Estimated that one must visit 25 Web
pages in order to read about 12 basic concepts about skin cancer
-
SENSEMAKING
SHOEBOX
EVIDENCE FILE
Search & Filter
Read & Extract
Schematize
Build Case
Tell Story
Search for Information
Search for Relations
Search for Evidence
Search for Support
Reevaluate
TIME or EFFORT
STRU
CTUR
E SCHEMAS
HYPOTHESES
PRESENTATION
EXTERNAL DATA
SOURCES
-
SENSEMAKING
SHOEBOX
EVIDENCE FILE
Search & Filter
Read & Extract
Schematize
Build Case
Tell Story
Search for Information
Search for Relations
Search for Evidence
Search for Support
Reevaluate
TIME or EFFORT
STRU
CTUR
E SCHEMAS
HYPOTHESES
PRESENTATION
EXTERNAL DATA
SOURCES
Sensemaking Loop
Foraging Loop
-
Entity Workspace Notebook
Drag-and-drop interface for capturing knowledge
Snap-together knowledge
Captures the user’s degree of interest
Controls automatic highlighting
-
AUGMENTED SOCIAL COGNITION
107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL Automatic Behavior 10-3 (msec) 10-4
Newell
Meg Stewart
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling) Attention-reactive displays (Implicit coupling) Sensemaking tools (Representation shift) SOCIAL LEVEL (Social cognition) Social sensemaking (Social representation
shift)
-
TAGGING WORKS IF MANY PEOPLE
1 User
1 User: 6 tags Many users: 100
tags Need ~ 20
Furnas
-
Info
rmat
ion
Con
nect
ivity
Social Connectivity
-
Nova Spivack
-
Ben Wattenberg
-
Viégas & Wattenberg
MANY EYES
-
Social Amplification
Rep
rese
ntat
ion
Am
plifi
catio
n
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling) Attention-reactive displays (Implicit coupling) Sensemaking tools (Representation shift) SOCIAL LEVEL (Social cognition) Social sensemaking (Social representation
shift) Input on social context (Social representation
shift)
-
WikiDashboard Social Transparency: Make socially significant information visible.
Bongwon Suh & Ed Chi
-
lowering system costs increases sharing
Ma.gnolia
Ma.gnolia
Google Reader
Media Wiki Google Notebook
Google Notebook
intended recipient:
-
COGNITIVE LEVEL (Immediate behavior) Minimize mental time and motion (Efficient communication) Input on output (Representation shift) RATIONAL LEVEL (Problem solving) Maximize (information gain)/(time cost) (Efficient search) Information scent (by design) (Efficient semantic
search) Information scent (by machine) (Semantic coupling) Information visualization (Visual representation
shift) Input on data (Data coupling) Attention-reactive displays (Implicit coupling) Sensemaking tools (Representation shift) SOCIAL LEVEL (Social cognition) Social sensemaking (Social representation
shift) Input on social context (Social representation
shift)
-
Efficient Communication Tight Coupling
Representation Shift
-- Minimize mental time and motion -- Maximize (information gain/(time cost) -- Information scent (by design)
-- Input on data -- Implicit coupling -- Information scent (by machine)
-- Input on output -- Information visualization -- Sensemaking tools -- Social sensemaking