Usability Testing Kathryn Summers ©2010
-
date post
18-Sep-2014 -
Category
Documents
-
view
809 -
download
0
description
Transcript of Usability Testing Kathryn Summers ©2010
Usability TestingUsability Testing
Kathryn Summers©2010
Copyright 2003 Kathryn Summers
Usability TestingPreparation
RecruitingScreeningTasks and Scenarios
TestingFacilitationObservation
AnalysisPresentation
Benefits to User-Centered Products (2/09)
Users find them more useful (can do more tasks that the user cares about)
Users can do things more efficientlyUsers learn how to use them more easily
and use more oftenUsers like them more (less frustrating,
more enjoyable
Costs of Poor Design
Copyright 2003 Kathryn Summers
Costs to clientincreased support costsreduced salesexpensive redesign
Costs to customer/userreduced employee productivityincreased employee frustrationmore frequent mistakes
Some general usability principles
Copyright 2003 Kathryn Summers
Simple and natural language (user’s language and labels)
visual consistency (graphics, navigation, layout, labeling)
appropriate feedbackerror prevention, error recoveryhelp information
And information design principles
Copyright 2003 Kathryn Summers
Information available right when and where it’s needed
Organization and presentation of information reflects user’s needs and goals
Visual design reinforces task flows and logical information groupings
Types of tests: Evaluative Testing
Copyright 2003 Kathryn Summers
Confirm that you’ve met a benchmark for usability before release
Confirm that product is more usable than a prior version or a competing product
Evaluative testing will focus on tasks and may often involve quantitative measurements
Types of tests: Exploratory testsTesting prototypes, or comparing prototypes
Copyright 2003 Kathryn Summers
Focus on exploring relationship between system image and user’s mental modelRepresenting classes of objectsRepresenting relationships between objectsAllowing user to manipulate objects
Test navigation, help access, subject matter organization
Values of UsabilityReal world/real user perspectiveTends to find serious functional issues
Values of Eye-trackingAllows testers to see if users read or not –
amount of time spent in area of screen vs scanning
Has value in decisions on layout/ navigation/labeling
Effective user testing
Copyright 2003 Kathryn Summers
Testing the right peopleTesting the right tasks
User testing vs. market research vs. “testing the software”
Watching with attention
Copyright 2003 Kathryn Summers
What users expect from a designHow they respond
Most understandable ways to present content: How the website should be structuredExact function,
arrangement of function, and system
Does the website actually work for the people who are supposedly going to use it? Why or why not?
Determine who is coming online in your targetDemographicPsychographicAttitudes and preferences
Determine what information will provide value to your targetDistinct information needs
and preferences by target sub-segment
Information gaps and priorities
Tools and resources
Market ResearchUser Research
Preparation: Recruiting Participants
Identify your target user groups (demographics, background knowledge, experience, relationship to site/tool, goals)
Decide which groups you want, and how many users you’ll recruit
(decide what characteristics users will share, what characteristics will vary)
Draft a test participant screener (what are make/break characteristics)
Test the screener
Recruiting Participants Decide on number of participants Over recruit (dropout/no show rate) Determine compensation
(amount, cash vs. gift, depends on time spent, organization type, project type)
Acquire consent
Preparation: Developing the Test Identify test objectives/create list of problems Translate objective/problem list into tasks Prioritize tasks (frequency, importance), decide
which tasks to test, Develop scenarios for tasks Write test script (scenarios) Identify resources participants will need for the
tasksHardware, software, data files, instructions, internet connection, time
Determine what data/measurements to collect (metrics)
Conduct a pilot test
Tasks vs. Scenarios Tasks are series of actions made to
achieve goal – ie., “request article from Interlibrary Loan using online form.”
Scenarios are tasks put into a short narrative – ie., “your instructor recommends an article and gives you a citation for “Usability Tests made Simple,” from The Journal of Usability, 2 (34), 11-23. You realize that Cook Library doesn’t have that journal, and you would like to request it from Interlibrary Loan”.
They are meant to take some of the artificiality out of the task.
Tasks vs. ScenariosScenarios should: Allow user to SHOW not TELL actions Include one and only one task Have defined set of pathways to complete
task Use natural language (user’s, not
system’s) Be short – avoid excessive background,
irrelevant details Be clear and provide enough instruction
detail to complete the task (test scenarios with pilot users)
Tasks vs. ScenariosScenarios should NOT: Walk user through process
(e.g. You have the afore-mentioned article citation. Go to the Interlibrary Loan link located in the list of services on the left. Complete the online form including author, article title, journal title, volume, issue and page number)
Scenario should test user’s ability to complete the task in a natural way without step-by-step instructions.
Preparation: Writing scriptWhy a script?
Ensures consistencyScript should include:
Explanation of testConsent (if not done beforehand)Think aloud protocolTasks (description if written, scenarios if
verbal)Questionnaire/demographic questionsFollow up if appropriate
Preparation: Establishing rolesFacilitator
ScriptObserver(s)
Actions Metrics CommentInferences
Observers can capture metrics in metrics-based testing
Copyright 2003 Kathryn Summers
time on taskerror ratesuccessful completion ratenumber of times (or how long) help system
is accessed time spent recovering from errorsnumber of commands/features used
Preparation: Pilot testTest script to see if scenarios/tasks are
workingTest equipmentPractice roles for moderator and observers
Preparation: MaterialsParticipant ListConsent formsCompensation/acknowledgementsScriptsTask scenarios (written/part of script)Observation forms
Facilitating the test Explain the test Help the test participant feel comfortable
(emphasis on testing SYSTEM not USER) Model/Encourage a think-aloud protocol Listen and observe – avoid “rescuing” Probe (avoid using leading questions) Respect user’s expertise
Becoming a better moderator
Copyright 2003 Kathryn Summers
Become a “people” personBe calm and flexiblePay close attentionWatch details but within larger context
Cohesive picture of each testPatterns between tests
Don’t just rely on memoryCommunicate results effectivelyStay organizedPractice!!!
Watching with attention
Copyright 2003 Kathryn Summers
Try not to test your own forms/designsTry to watch with a completely open mindWatch carefully—pay attention to body
language, facial expressions, sounds, where the user is looking, and what they do
Pay most attention to behavior, then to explanations, least attention to preferences or predictions about what other users want
Most of all…
Copyright 2003 Kathryn Summers
Understand that the user never makes “mistakes”—the user only makes efforts to solve the problem
Your goal is to understand what problem-solving effort the user is making
Analysis during the test: Keep a problem list
Copyright 2003 Kathryn Summers
Keep a free-hand list during the test—note “critical incidents” that you may ask follow-up questions about at the end of the session
Use the “highlighter” methodHave a printout of interface screens on hand; circle problem areas as you observe them during the tests
Watch the tapes, categorizing related problems after the test
Capture metrics if desired
ReferencesGoldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., Wichansky, A. M. (2002).
Eye tracking in web search tasks: Design implications. In Proceedings of the 2002 symposium on Eye tracking research & applications (pp. 51-58). Retrieved March 28, 2009 from ACM Digital Library.
Hackos, J. T. & Redish, J. C. (1998). User and task analysis for interface design. New York: John Wiley & Sons.
Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. M. (1991). User interface evaluation in the real world: Comparison of four techniques. In Proceedings of the SIGCHI conference on human factors in computing systems: Reaching through technology (pp. 119-124). Retrieved March 28, 2009 from ACM Digital Library.
Molich, R., Ede, M. R., Kaasgaard, K., Karyukin, B., (2004). Comparative usability evaluation. Behavior & Information technology, 23 (1), 65. 74. Retrieved March 28 from Academic Search Premier database.
Rubin, J. (1994). Handbook of usability testing: How to plan, design, and conduct effective tests. New York: John Wiley & Sons.
Snyder, C. (2003). Paper prototyping: The fast and easy way to design and refine user interfaces. San Francisco: Morgan Kaufman.
Limitations of usability testing
Copyright 2003 Kathryn Summers
Testing situations are always artificial (maybe in a lab, assigned tasks, paid participation)
Test participants are rarely perfectly representative users
The test design can never fully duplicate natural user behaviorResults depend heavily on tasks—we see
what we think to look for
Finding participants with lower literacy skills
Copyright 2003 Kathryn Summers
Try to recruit participants that Have a strong motivation to care about the
interface (e.g., users with a particular health condition)
Read at 8th grade level or below (REALM 60 or below)
Have been observationally screened for a minimum level of computer ability Open browserNavigate to website using address bar OR
searchScrollUse linksUse mouse
Where to recruit users with lower literacy skills
Copyright 2003 Kathryn Summers
Recruit fromHealth clinicsLiteracy centersPublic libraries
Plan for a two-step recruiting approachAdminister the REALM; observe computer useSchedule selected users for testing
REALM (Rapid Estimate of Adult Literacy in Medicine) (Davis, et al., Family Medicine 1993 25.6: 391-5.)