Predictive Evaluation Simple models of human performance.
-
date post
22-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Predictive Evaluation Simple models of human performance.
Predictive Evaluation
Simple models of human performance
Recap
I. Senses A. Sight B. Sound C. Touch D. Smell
II. Information processing A. Perceptual B. Cognitive 1. Memory a. Short term b. Medium term c. Long term 2. Processes a. Selective attention b. Learning c. Problem solving d. Language
III. Motor system A. Hand movement B. Workstation Layout
Simple User Models
Idea: If we can build a model of how a user works, then we can predict how s/he will interact with the interfacePredictive model predictive
evaluation No mock-ups or prototypes!
Two Types of User Modeling
Stimulus-Response Hick’s law Practice law Fitt’s law
Cognitive – human as interpreter/predictor – based on Model Human Processor (MHP) Key-stroke Level Model
• Low-level, simple GOMS (and similar) Models
• Higher-level (Goals, Operations, Methods, Selections)• Not discussed here
Power law of practice
Tn = T1n-a
Tn to complete the nth trial is T1 on the first trial times n to the power -a; a is about .4, between .2 and .6
Skilled behavior - Stimulus-Response and routine cognitive actions
• Typing speed improvement• Learning to use mouse• Pushing buttons in response to stimuli• NOT learning
Power Law: Tn = T1n-a
n T_n1 52 3.795 2.62
10 1.9915 1.6925 1.37
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
n
Trial Time
If first trial (T1) takes 5 seconds, how long will future trials take? When will improvements level off? (a = -0.4)
Uses for Power Law of Practice Use measured time T1 on trial 1 to predict
whether time with practice will meet usability criteria, after a reasonable number of trials How many trials are reasonable?
Predict how many practices will be needed for user to meet usability criteria Determine if usability criteria is realistic
Hick’s law
Decision time to choose among n equally likely alternatives T = Ic log2(n+1)
Ic ~ 150 msec
How can we use this? Explanation on how to calculate base 2
logs:http://mathforum.org/library/drmath/view/55613.html
Uses for Hick’s Law
Menu selection Which will be faster as way to choose from
64 choices? Go figure: Single menu of 64 items Two-level menu of 8 choices at each level Two-level menu of 4 and then 16 choices Two-level menu of 16 and then 4 choices Three-level menu of 4 choices at each level Binary menu with 6 levels
Fitts’ Law
Models movement times for selection (reaching) tasks in one dimension
Basic idea: Movement time for a selection taskIncreases as distance to target
increasesDecreases as size of target increases
Original Experiment
1-Dd w
Components
ID - Index of difficulty
larger target => more information (less uncertainty)
ID = log2 (d/w + 1.0)
bits result
width (tolerance)of target
distance to move
Components
MT - Movement time
MT is a linear function of ID k1 and k2 are experimental constants
MT = k1 + k2*IDMT = k1 + k2 *log2 (d/w + 1.0)
Exact Equation
Run empirical tests to determine k1 and k2 in MT = k1 + k2* ID
Will get different ones for different input devices and device uses
MT
ID = log2(d/w = 1.0)
Questions
What do you do in 2D?
h x l rect:one way is ID = log2(d/min(w, l) + 1)
Should take into account direction of approach
Uses for Fitt’s Law
Menu item size Icon size Scroll bar target size and placement
Up / down scroll arrows together or at top and bottom of scroll bar
Example: what would Fitt’s say about multi-level menus? What about pop-up menus?
Keystroke-Level Model (KSLM) KSLM - developed by Card, Moran &
Newell, see their book* and CACM* The Psychology of Human-Computer
Interaction, Card, Moran and Newell, Erlbaum, 1983
Skilled users performing routine tasks Assigns times to basic human operations -
experimentally verified Based on MHP - Model Human Processor
KSLM Accounts for
Keystroking TK
Mouse button press TB
Pointing (typically with mouse) TP
Hand movement betweenkeyboard and mouse TH
Drawing straight line segments TD
“Mental preparation” TM
System Response time TR
Using KSLM - Step One
Decompose task into sequence of operations - K, B, P, H, D (no M operators yet; R can be used always or not at all)
Step One : MS Word Find Command Use Find Command to locate a six character
word H (Home on mouse) P (Edit) B (click on mouse button - press/release) P (Find) B (click on mouse button) H (Home on keyboard) 6K (Type six characters into Find dialogue
box) K (Return key on dialogue box starts the find)
Using KSLM - Step Two
Place M operatorsRule 0a. In front of all K’s that are NOT part of
argument strings (ie, not part of text or numbers)
Rule 0b. In front of all P’s that select commands (not arguments)
Step Two : MS Word Find Command
H (Home on mouse)MP (Edit)B (click on mouse button)MP (Find)B (click on mouse button)H (Home on keyboard)6K (Type six characters)MK (Return key on dialogue box
starts the find)
Rule 0b: Pselects command
Rule 0b: Pselects command
Rule 0a: Kis argument
Using KSLM - Step 3
Remove M’s according to heuristic rules (Rules relate to chunking of actions)Rule 1. Anticipated by prior operation
– PMK ->PK (point and then click is a chunk)
Rule 2. If string of MKs is a single cognitive unit (such as a command name), delete all but first
– MKMKMK -> MKKK (same as M3K) (type “run rtn is a chunk)
Rule 3. Redundant terminator, such as )) or rtn rtnRule 4. If K terminates a constant string, such as command-rtn, then
delete M• M2K(ls)MK(rtn) -> M2K(ls)K(rtn) (typing “ls” command in Unix
followed by rtn is a chunk)
H (Home on mouse)MP (Edit)B (click on mouse button)MP (Find)B (click on mouse button)H (Home on keyboard)6K (Type six characters)MK (Return key on dialogue box
starts the find)
Step 3 : MS Word Find Command
Rule 4 Keep M
Rule 1 delete MH anticipates P
Using KSLM - Step 4
Plug in real numbers from experiments K: .08 sec for best typists, .28 average, 1.2
if unfamiliar with keyboard B: down or up - 0.1 secs; click - 0.2 secs P: 1.1 secs H: 0.4 secs M: 1.35 secs R: depends on system; often less than .05
secs
Step 4 : MS Word Find Command
H (Home on mouse)P (Edit)B (click on mouse button - press/release)MP (Find)B (click on mouse button)H (Home on keyboard)6K (Type six characters into Find dialogue box)MK (Return key on dialogue box starts the find) Timings
H = 0.40, P = 1.10, B = 0.20, M = 1.35, K = 0.28 2H, 2P, 2B, 2M, 7K
Predicted time = 8.06 secs
Example: MS Windows Menu Selection
Get hands on mouse Select from menu bar with click of
mouse button The “pull down” menu appears Select desired item from the pull down
menu
Step 1: MS Windows Menu
H (Home on mouse)
P (point to menu bar item)
B (left-click with mouse button)
P (point to menu item)
B (left-click with mouse button)
Step 2: MS Windows Menu - Add M’s
H (get hand on mouse)
MP (point to menu bar item)
B (left-click with mouse button)
MP (point to menu item)
B (left-click with mouse button)
Rule 0b: Pselects command
Rule 0b: Pselects command
Step 3: MS Windows Menu - Delete M’s
H (get hand on mouse) MP (point to menu bar item) B (left-click with mouse button) MP (point to menu item) B (left-click with mouse button)
Keep M
Rule 1 Manticipated by P
Step 4: MS Windows Menu Calculate Time H (get hand on mouse) P (point to menu bar item) B (left-click with mouse button) MP (point to menu item) B (left-click with mouse button) Textbook timings (all in seconds)
H = 0.40, P = 1.10, B = 0.20, M = 1.35 H, 2P, 2B, 1 M
Total predicted time = 4.35 sec
Macintosh Menu Selection
Operator sequence H(mouse)P(to menu item)B(down)PB(up)
Now place Ms H(mouse)MP(to menu item)B(down)MPB(up)
Selectively remove Ms H(mouse)MP(to menu item)B(down)MPB(up)
Textbook timings (all in seconds)• H = 0.40, P = 1.10, B = 0.10 for up or down, M = 1.35• H, 2P, 2 B, 1 M
Total predicted time = 4.15 sec Macintosh is predicted to be .2 secs faster than MS Windows, about 5%
Rule 0b
Rule 0b
Rule 1 Delete H anticipates P
KSLM Comparison Problem
Are keyboard accelerators always faster than menu selection?
Use MS Windows to compare Menu selection of File/Print (previous
example estimated 4.35 secs.) Keyboard accelerator
• ALT-F to open the File pull down menu• P key to select the Print menu item
Assume hands start on keyboard
KSLM Comparison:Keyboard Accelerator for Print
Use Keyboard for ALT-F P (hands already there) K(ALT)K(F)K(P)
MK(ALT)MK(F)MK(P)
MK(ALT)K(F)MK(P)• 2M + 3K = 2.7 + 3K
Times for K based on typing speed Good typist, K = 0.12 s, total time = 3.06 s Poor typist, K = 0.28 s, total time = 3.54 s Non-typist, K = 1.20 s, total time = 6.30 s Time with mouse was 4.35 sec
Conclusion: Accelerator keys not necessarily faster than mouse for all users!
First Kanticipatessecond K
Using KSLM
Skilled users Performing routine tasks
The user has done it many times before No real learning going on Some modest “thinking” as captured by Ms
Rules for placing Ms are heuristics Best use is for comparing alternatives
Sometimes predictions are off But rankings of faster - slower tend to be
accurate
Now You Get to Do It
KSLM of the hierarchical menu selection exampleCombine with Hick’s Law
Draw through text and make it boldBy pointing to BOLD icon in floating
paletteBy selecting BOLD from pull-down
menu
Cognitive models - many flavors
More complex than KSLMHierarchical
GOMS - Goals, Operators, Methods, SelectorsCCT - Cognitive Complexity Theory
LinguisticTAG - Task Action GrammarCLG - Command Language Grammar
Cognitive architecturesSOAR, ACT