Evaluating User InterfacesWalkthrough Analysis

Joseph A. Konstan

konstan@cs.umn.edu

CSci 5115 2

October 10 Introduction to Evaluation Cognitive Walkthrough Other Evaluation Methods

CSci 5115 3

Interface Development Methodology Prototype and Iterate

keep iterating until it is good enough

evaluate along the way to assess What is Good? What is Good

Enough? set usability goals should relate to tasks

CSci 5115 4

Casual Iteration

Find major usability problems missing features user confusion poor interaction

Try interface with specific tasks first use designers, then move

towards users observe overall usage

CSci 5115 5

Casual Iteration

Remember the goal don’t defend the interface don’t bias the tests towards the

interface If possible, allow user exploration

may even lead to capturing new tasks

Consider alternative ways to fix a problem

CSci 5115 6

Limits of Casual Iteration Does not indicate when to stop Financial trade-offs Justification of delay

CSci 5115 7

Usability Goals and Measures Concrete, quantitative measures of

usability learning time use time for specific tasks and users error rates measures of user satisfaction

Comparative usability goals compare with prior versions or

competitors

CSci 5115 8

Things to Watch

Goals should be realistic 100% is never realistic

Many goals go beyond the application UI training, manuals

Testing goals should help improve the UI detail--not just good/bad

CSci 5115 9

Exercise:Setting Usability Goals In project groups, come up with 2

usability goals for your project discuss the feasibility of testing these

goals what is needed for the test when in the process can they be

tested? how much effort, user

preparation/training, etc.? what would you learn from the test?

CSci 5115 10

Interface Evaluation

Goals of interface evaluation find problems find opportunity for improvement determine if interface is “good

enough”

CSci 5115 11

With or Without Users

Users are expensive and inconsistent usability studies require several

users some users provide great

information, others little Users are users

cannot be simulated perfectly Best choice--Both

CSci 5115 12

Evaluation Without Users Quantitative Methods

GOMS/keystroke analysis back-of-the-envelope action

analysis Qualitative Methods

expert evaluation cognitive walkthrough heuristic evaluation

CSci 5115 13

Walkthrough Analysis

Economical interface evaluation low-fidelity prototype development team

users optional Effective, if

goal is improvement, not defense some team members skilled proper motivation

CSci 5115 14

Cognitive Walkthrough

Goals imagine user’s experience evaluate choice-points in the interface detect confusing labels or options detect likely user navigation errors

Start with a complete TCUID scenario never try to “wing it” on a walkthrough

CSci 5115 15

Tell a Believable Story

How does the user accomplish the task

Action-by-action Based on user knowledge and

system interface

CSci 5115 16

Best Approach

Work as a group don’t partition the task

Be highly skeptical remember the goal!

Every gap is an interface problem

CSci 5115 17

Who Should Do the Walkthrough Designers, as an early check Team of designers & users

remember: goal is to find problems

avoid making it a show Skilled UI people may be valuable

team members

CSci 5115 18

How Far Along

Basic requirements description or prototype of interface know who users are (and their experience) a task description a list of actions to complete the task

(scenario) DO NOT try to create the action list on the fly!

Viable once the scenario and interface sketch are completed

CSci 5115 19

How to Proceed

For each action in the sequence tell the story of why the user will do it ask critical questions

will the user be trying to produce the effect? will the user see the correct control? will the user see that the control produces the

desired effect? will the user select a different control instead? will the user understand the feedback to

proceed correctly?

CSci 5115 20

Walkthroughs are not Perfect

They won’t find every problem limited by nature

new users who know what task they need to accomplish

biased towards correct action sequence limited in implementation

hard to shed the expertise of evaluators A useful tool in conjunction with others

CSci 5115 21

Exercise: Cognitive Walkthrough Analysis In non-project groups of 3-5 Users and Task to be announced Scenario developed jointly Perform walkthrough

identify problems estimate error probabilities (25%

intervals) Remember who your users are!

CSci 5115 22

GOMS/Keystroke Analysis Formal action analysis

accurately predict task completion time for skilled users

Break task into tiny steps keystroke, mouse movement,

refocus gaze retrieve item from long-term memory

Look up average step times tables from large experiments

CSci 5115 23

GOMS/Keystroke Analysis Primary utility: repetitive tasks

e.g., telephone operators benefit: can be very accurate (within

20%) may identify bottlenecks

Difficulties challenging to decompose accurately long/laborious process not useful with non-experts

CSci 5115 24

Back-of-the-Envelope Action Analysis Coarse-grain

list basic actions (select menu item) each action is at least 2-3 seconds what must be learned/remembered? what can be done easily? documentation/training?

Goal is to find major problems Example: 1950’s 35mm camera

CSci 5115 25

Expert Evaluation

Usability specialists are very valuable double-specialists are even better

An inexpensive way to get a lot of feedback

Be sure the expert is qualified in your area

CSci 5115 26

Looking Ahead

Next week: Heuristic Evaluation Walkthroughs Due

“raw” notes• notes from each step of walkthrough• copy of prototype used, markups• copy of scenarios used (note changes

or fixes)

processed results• 1-2 pages of issues identified,

solutions not needed