Usability Testing Basics: Remote and In-Person Studies
-
Upload
uxpa-international -
Category
Technology
-
view
453 -
download
1
Transcript of Usability Testing Basics: Remote and In-Person Studies
![Page 1: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/1.jpg)
Getting Started with Usability TestingInstructors: Thyra Rauch, John Schrag, and Carol SmithUXPA 2016
![Page 2: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/2.jpg)
Schedule
05:30 Introductions, Agenda, Materials 05:40 Measurements & Testing 06:05 Activity I 6:25 Planning Test Logistics 6:40 Activity II 7:00 Selecting & Preparing Materials 7:10 Interacting with Test Participants 7:30 Break 8:00 Activity III 8:20 Testing Experiences 8:25 Activity IV and Team Presentations 9:10 Team Presentations 9:25 Final Q&A and Wrap-Up 9:30 Adjourn
2
![Page 3: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/3.jpg)
Introduction
Instructor introductions
Audience experience
3http://creativecommons.org/licenses/by-sa/2.0/http://www.flickr.com/photos/raphaelquinet/513351385/sizes/l/in/photostream/http://www.flickr.com/photos/raphaelquinet/
![Page 4: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/4.jpg)
Why Test?
You are not your user May miss details; too close to design Unforeseen requirements
Validate understanding of tasks and context
More satisfied users Less training needed Fewer services calls
4
![Page 5: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/5.jpg)
Myths of Usability Testing
Need a labTesting occurs at the end of the dev cycle
Must test everythingNeed to test 100 peopleA “good” test means no changesUsability is “system testing”, “user-acceptance testing”, etc.
Need a large test team, each playing a specific role/part
5
![Page 6: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/6.jpg)
Just Do It!
Make it realisticAny locationAny stage of development (earlier is better)
Start small Test an idea (not a feature) Just a few users
Anytime (un-moderated)
6
![Page 7: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/7.jpg)
Getting Started
7
![Page 8: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/8.jpg)
Plan and Prepare for the Test
• Scope the effort–Budget, resources, goals, time–Put together a test team
•Design the test–Goals, participants, tasks, measures, methodology, team roles etc.–Summative or formative?
8
![Page 9: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/9.jpg)
Testing Plan
Goals What testing and how to measure success
Test protocol – remote or in person?
User groups and participants
Recruiting/incentives Scenarios
Share with team Keep concise
9
![Page 10: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/10.jpg)
Create Test Scenarios
Representative of typical tasksDescribe what, not how
“You want to buy a shirt for your father. Find a man’s blue dress shirt for under $40.”
“As a systems administrator for your company, you want to install, configure, and test the xyz application.”
Avoid leading wording (e.g., actions)
10
![Page 11: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/11.jpg)
Measurements
Success Time on task % of tasks completed/not completed
Number of steps to accomplish task
Learning time Number of errors Number of times help consulted
Satisfaction
11Photo By Kenyaboy7Andrew Chipley http://www.flickr.com/photos/kenyaboy7/3749535540/
![Page 12: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/12.jpg)
Facilitator Role
Conducts test and manages time and frustration
Communicates with participant (before, during, after)
Prompts for more informationEnsures legal requirements are met (NDAs, recording permissions)
12
![Page 13: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/13.jpg)
Additional Roles
Observer(s)* Takes careful notes Observations (notes inferences as well)
Exact quotes Context
“Computer” (paper prototyping, wizard of oz)
Mimics actions of computer
13*See guidance for Observers in Resources section
![Page 14: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/14.jpg)
Testing Protocols
Think-aloudActive interventionCo-discoveryRetrospective
14
![Page 15: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/15.jpg)
Activity I
15
![Page 16: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/16.jpg)
Basic Usability Study
Select item to testDiscover stakeholder’s needsHigh level goalsMeasurements to be madeHigh level scenario (sentence or two)
16
![Page 17: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/17.jpg)
Planning Test Logistics
17
![Page 18: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/18.jpg)
Recruit Participants
Use a participant screening profileTwo weeks to complete recruit (average)
Recruiters Marketing or product marketing department
Professional recruitment agency Members of your team or yourself
18
![Page 19: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/19.jpg)
Run a Pilot Study
Discover problems with study or concept being tested
Estimate time needed for the testRefine script and tasks for testVerify if tasks are typical (is it something users actually do?)
Practice before going live with participants
Get new ideas for follow-on questions or things to observe
19
![Page 20: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/20.jpg)
Do I need a lab?
20Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994.
Computer / Concept
FacilitatorParticipant
Observer
Timer Logger
![Page 21: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/21.jpg)
Portable Labs
Temporary setup Capture screen, video and audio
Complicates observation
Anywhere (conference room, remotely)
21Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/
![Page 22: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/22.jpg)
Remote Testing
Moderated and AutomatedCapture screen, video and audio (not all automated collect this)
Moderated: Connect to both computer and phone
Technology Bandwidth, operating system, browser Test setup and connections ahead of time
22
![Page 23: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/23.jpg)
Activity II
23
![Page 24: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/24.jpg)
Recruiting Profile
Key elements Mimic population using product
Critical factors that make a difference Age Experience
How they will be screened Phone Online Follow-up and scheduling
24
![Page 25: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/25.jpg)
Recruiting Profile
Samples Why decisions made What going after
Create recruiting profile for participants for usability study designed in Activity 1
25
![Page 26: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/26.jpg)
Selecting & Preparing Materials
26
![Page 27: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/27.jpg)
Why Use a Script?
Promotes consistency in conducting the test
Reminds to welcome and thank peopleReminds to reset/configure between participants
Ensures NDAs and other legal requirements are not missed
Prepare for non-native language speakers
Tell me more…
27
![Page 28: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/28.jpg)
Parts of a Script
Welcome to participantsSteps in test (forms, tutorials, tasks, questions)
Thank you to participants (and incentives if any)
Notes to yourself Reset/configuration prompts
28
![Page 29: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/29.jpg)
Test Folders
Organize materials per participant
Easily access individual results
Store as a groupPrepare a folder for each participant
All test materials(script, forms, printed tasks etc.)
Don’t use names- label folders: Pilot, P1, P2, P3
29
![Page 30: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/30.jpg)
Informed Consent
Your responsibility to inform participant
Every time Ethical and legal responsibility Do no harm (physical, emotional)
Explain consent form* Plain language – what, why, stop anytime, etc.
Participant signs (Google form remotely)Rarely lose participants over form
30*Examples in Appendix
![Page 31: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/31.jpg)
Interacting With Test Participants
31
![Page 32: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/32.jpg)
Welcome
Make sure the participant feels welcomed and appreciated.
Stress that you are not testing them, rather you are testing…
Encourage feedback, positive or negative
“I was not involved in the design of this so you can’t hurt my feelings”
32
![Page 33: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/33.jpg)
Facilitating
Follow your scriptFrame questions in unbiased mannerRemain passive in facial expressions and body language as much as possible while participant is working
Don’t lead or give clues (unless frustration is clear and evident)
Probe don’t demandGently prompt during think-aloud protocol if necessary
33
![Page 34: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/34.jpg)
Probes
Ask open-ended questionsEncourage participant to elaborateUse participant’s wordsListen for vocalizations and watch non-verbal gestures
Have participant tell you when they are done
Give them time to think
34
![Page 35: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/35.jpg)
Example Phrases
How does this compare with your expectations?
What specifically are you referring to?What are you looking for?Tell me more…Can you give me an example?
35
![Page 36: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/36.jpg)
Common Issues
Refusal to critique Ask preference and to compare interfaces Watch what they DO, not what they SAY
Taking control - diverting to their issue Let them see you are listening and writing it down
If no resolution, thank and end session
- give them the incentive.
36
![Page 37: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/37.jpg)
Getting Users Unstuck
It’s not a question of “if” but “when” Users might be embarrassed at what they perceive to be their fault
Users may not be able to articulate what they don’t understand
Don’t directly answer initial questions
37
![Page 38: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/38.jpg)
Getting Users Unstuck
Progressive assistance What are you trying to do right now? What do you think the next step is? What would you do in this situation if you were at work?
Hints Do you see anything that might help you? (small hint)
Have you checked the Help? (medium hint)
What do you think the xxx button does? (large hint)
End before frustrated, or take a break
38
![Page 39: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/39.jpg)
Debriefing
Verbal and written questions What did you like best? What was the worst thing? What can be improved? Describe in 3 words Follow up to specific items noted during testing
Ratings Ease of use Willingness to use
Thank participant
39
![Page 40: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/40.jpg)
Activity III
40
![Page 41: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/41.jpg)
Probes – Group Exercise
Are you thinking of using the search feature?
Are you trying to find the price of the book?
41
![Page 42: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/42.jpg)
Alternatives
Are you thinking of using the search feature? (biased)
Can you tell me what you are thinking right now? (unbiased)
Are you trying to find the price of the book? (biased)
What are you trying to do? (unbiased)
42
![Page 43: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/43.jpg)
Rational
Can’t know what participant is thinking
Keep questions brief and clearAvoid
Asking participants to predict the future Putting “words in their mouth” Showing bias or hinting at your opinions
43
![Page 44: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/44.jpg)
Probes – Team Exercises
Break into teamsEach get a set of biased probesGenerate unbiased alternativesPresent/discuss with entire group
44
![Page 45: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/45.jpg)
Testing Experiences
45
![Page 46: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/46.jpg)
Low-fidelity/Paper Prototypes
Includes wireframesGood time to get initial feedback on concept, word choices and placement
Participants May not be able to understand May not be act the same as online
Always test again later in development
46
![Page 47: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/47.jpg)
High Fidelity Prototypes
Ideal time for testing – early enough in development, but participants can “kick the tires”
May be difficult to test with (broken links, undeveloped material, etc.)
47
![Page 48: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/48.jpg)
Functioning Products
Great for redesign projects – prior to start
Get baseline and goals for improvements
Reduce task sequence from 10 clicks to 5 Reduce time to perform task from 10 min to less than 3 min
Increase satisfaction from 4 to 5 on a 5-pt scale
Use existing data (Analytics, Help Desk, etc.) to create tasks and inform study
48
![Page 49: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/49.jpg)
Reporting
What was tested, how and whySuccess and Failure and whyRecommendationsFuture testing – benchmarking Just enough to communicate – short & sweet
49
![Page 50: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/50.jpg)
Activity IV
50
![Page 51: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/51.jpg)
Activity 4
Mock up (15 minutes)Round 1 of testing (5 minutes)Revise mock ups (10 minutes)Round 2 of testing (5 minutes)Prepare team presentation (10 minutes)
51
![Page 52: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/52.jpg)
Hotel Reservation Kiosk
Design and develop a new hotel check-in kiosk that can be installed in airports
Hotel chain wants to make it easy for their customers:
Check into their room at the airport Request a particular room type, if available
Request transportation to the hotel
52
![Page 53: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/53.jpg)
Ideal Scenario
During your search for a local phone book, you notice a hotel chain’s digital Kiosk, and you decide to try it.
You need to: Reserve a Hotel Room for the evening. (A room with 1 bed for 2 people, non-smoking)
Request transportation to the Hotel Reserve transportation from the hotel back to the airport
53
![Page 54: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/54.jpg)
Alternative Scenario
You want to go to the hotel right away to change before going to a fancy restaurant downtown.
You need to: Know if the hotel provides transportation. Reserve a room with a King-Size bed because you're fat and you want a view of the ocean.
Make sure that your “friend” who is coming tomorrow will have a room available next to yours.
54
![Page 55: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/55.jpg)
Team Presentations
55
![Page 56: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/56.jpg)
Final Q&A Questions?Handouts
56
![Page 57: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/57.jpg)
Survey Please complete the session survey:www.uxpa2016.org/survey/69
57
![Page 58: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/58.jpg)
Contact Us
Thyra Rauch IBM [email protected]
+1 (408) 463-2465
Carol Smith [email protected]
+1 (412) 212-8737@carologic
58
John SchragAutodesk [email protected]
@jvschrag
![Page 59: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/59.jpg)
Resources
59
![Page 60: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/60.jpg)
Usability by the numbers…
80% of SW life cycle costs occur during maintenance
80% of maintenance costs are due to unmet or unforeseen user requirements
Only 20% are bugs or reliability problemsTypical software program released with 40 usability design problems
Over 70% of CIO's state that one of their biggest problems is communicating with users to understand their needs
60
![Page 61: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/61.jpg)
Number of Participants
• Contentious issue• Nielsen - 5-7
participants uncover ~80% of major usability problems • IF representative • IF doing
representative tasks. • Testing more =
significant diminishing return.
• Virzi and Carol Barnum • Small number of
users is sufficient.
61http://www.useit.com/alertbox/20000319.htmlJakob Nielsen’s Alertbox. Why You Only Need to Test with 5 Users. March 19, 2000.
Number of Test Participants
![Page 62: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/62.jpg)
Recruitees
Could be Customers General consumers Members of organization or company
Difficulty of recruiting Specificity of screener Availability/interest in study (doctors vs. consumers)
Experience (highly experienced vs. novice)
62
![Page 63: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/63.jpg)
Recruiting Considerations
Accessibility of test location Include people with disabilities “We are all only temporarily able-bodied. Accessibility is good for us all.”
Language interpretersRemote and on-site studies
Technology availability Administrative rights Internet connectivity
63Quote by @mollydotcom at #stirtrek 2011 via @carologic
![Page 64: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/64.jpg)
Data Collection Forms
Customize for each studySpace for writing notes – each scenario spelled out.
Include space for tickmarks regarding easily measurable activities.
E.g. Participant accesses video clip E.g. Participant selects advertisement
64
![Page 65: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/65.jpg)
Tool Considerations
In person or remote? Lab or on-site?Prototype limitations (can it be online?, is it a document or a clickable site?)
Number of observers, number of participants?
Number of facilitators?Logging and video editing needs (time on task, highlight video creation)?
Surveys before or after?Eye tracking?
65
![Page 66: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/66.jpg)
Usability Testing Software
Morae OvoSilverBack (Mac only)Loop11 (Remote only)Tobii (Eye-tracker)SMI (Eye-tracker)SurveyMonkey
66
![Page 67: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/67.jpg)
Screen Sharing Software
GoToMeeting – http://www.gotomeeting.com
Lotus Sametime Unyte – http://www.unyte.com
YuuGuu -- http://www.yuuguu.com WebEx – http://www.webex.comYugma -- https://www.yugma.com/
Trouble Shooting: CoPilot - https://www.copilot.com/
67
![Page 68: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/68.jpg)
Satisfaction Questionnaires
Standard Usability Measurement Inventory (SUMI)
office/desktop software, purchase 50 questions
Website Analysis and MeasureMent Inventory (WHAMMI)
Purchase 20 questions
System Usability Scale (SUS) Free 10 questions
68
![Page 69: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/69.jpg)
Ethics Resources
Treatment of human subjects Web-based course Gives certificate of completion http://cme.cancer.gov/clinicaltrials/learning/humanparticipant-protections.asp
Guidelines for writing informed consent
http://ohsr.od.nih.gov/info/sheet6.htmlUXPA Code of Conduct
69
![Page 70: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/70.jpg)
NDA’s
Do I need a non-disclosure agreement (NDA)?
Maybe Legal department (be friendly) One-way, two-way*
AKA Confidentiality agreement (CA) Confidential disclosure agreement (CDA) Proprietary information agreement (PIA) Secrecy agreement (SA)
70
![Page 71: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/71.jpg)
Activity ______________________________________Who are you observing? (e.g. SME in soft-goods manufacturing, application admin)
What is the situation? ___________________________(e.g. phone interview, their work at their desk in person, remote usability study, etc.)
Date: __________________Time: __________________Location: ________________Note-Taker: ______________
People (P)Who are you observing? Who else is involved in the experience?
Objects (O) What objects do they interact with? What apps? Phone, etc.?
Environment (E) Characteristics of the setting, number/types of interruptions
Messages (M)What is communicated? How is it transmitted?
Services (S)What services are available to them?
Watch for these types of observations and put additional Questions/Ideas on the back. Content in this document described by Vijay Kumar in his book: 101 Design Methods
Physical What do they interact with?
CognitiveHow associate meanings? How learn?
SocialInteractions, decision making, scheduling, work?
CulturalWhat are shared norms, habits, values?
EmotionalWhat emotions are expressed and how?
What are your biggest takeaways?What did you learn? What surprised you?What is the participants need?
Interview & Observation Notetaking Form
![Page 72: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/72.jpg)
Observer RulesEveryone who observes a design session is asked to abide by a set of rules. The purpose of these rules is to minimize stress for the participants and to maximize the amount of information we get from the study.
Stay for the entire study Distractions are unhelpful and participants may get the impression that you’re leaving
because they’ve done something wrong (e.g. walking out in middle of a movie). If you can attend only part of a study, discuss with the facilitator beforehand to determine whether there is a way to accommodate this.
Don’t reveal information about the study to the participant It is often more useful to explore an area of difficulty in detail rather than try to “get
through” all the topics. The facilitator will track time so that we can cover as many of the important areas as possible.
Respect participants and the confidentiality of their data We have promised the participants that their participation is confidential. This means
that we should not include their names in any reports or other communication such as email, and we should refrain from discussing them by name outside the test setting.
Do not make negative comments about people—there is always a risk that a derogatory comment could be overheard or otherwise make its way back to the user.
Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved.
![Page 73: Usability Testing Basics: Remote and In-Person Studies](https://reader038.fdocuments.net/reader038/viewer/2022103010/587fa9ab1a28ab825e8b7a63/html5/thumbnails/73.jpg)
Observer Rules (continued) Remain silent and silence phones You may notice something so surprising that you are tempted to laugh or exclaim out loud.
This is not unusual. Unfortunately, participants might think you are laughing at them. Keep as quiet as possible. You will have opportunities to ask questions at the end. If necessary, pass a note to the facilitator.
Observe only - no questions or support If reviewing an interface, it’s likely participants will have problems, and it is normal to feel a
temptation to help. Please don’t. Instead, try to understand why it was that the user got stuck or went down the wrong path. It’s the facilitator’s role to get users back on track if they get really stuck. And if the facilitator poses a question during the test, he or she is asking the users, not you—please don’t answer unless the facilitator specifically directs a question to you.
Avoid “Design Questions” - Questions that ask the user their opinions about how to design aspects of the application (such as, “Where would you like to see these navigation buttons?”) can take a lot of time to answer and produce only limited results. Instead, focus on trying to understand the problem—we’ll come up with solutions later, outside the test.
In person, be conscious of your body language Although most studies are interesting, not every moment will be fascinating. If something is
happening that isn’t of interest to you but may be to others, sit quietly without fidgeting. Take notes to stay alert.
Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved.