Design usability tests to make data-driven design decisions by Teresa Washburn and John McGloon

© 2013 Teresa Washburn and John McGloon© 2013 Teresa Washburn and John McGloon

Design usability tests to make data-driven design decisions

John McGloonTeresa Washburn

© 2013 Teresa Washburn and John McGloon

Things we’ll cover

• Write better screeners by linking personas• Prioritize with task sampling and selection• Write a test script • Use various prototypes• Facilitate a test• Plan for observers• Incorporate additional methods (card sorts,

system usability scales)• Summarize a usability test



Step 1: Have a plan• What are your goals?• What does success look like?

– Meet with your stakeholders and define success• Why should you define success with

stakeholders?– Gets everyone on the same page– When observing, everyone should be coming from

the same position– After the test, there will be less chance of

miscommunication© 2013 Teresa Washburn and John McGloon


What are personas?

• Personas are a guide to help the team keep the user top of mind

• Audience and purpose• Make generalizations!



Persona map



Link personas to your screener!

• Write screener questions that only Taylor Biggs could answer

• Watch out for Taylor imposters• Constructing the right screener will get you

the right participant• Make better design decisions based on the

right set of participants



Prioritize with task sampling and selection

• How do you decide what to have the participant do? – What are your goals?

• Complete a task analysis

Goal

SubtaskSubtask

Task TaskTask



Task analysis: Checkout2.0

Scan products (repeat for ea. Item)

2.1Locate

barcode on

product

2.1.1No

barcode located

2.2Place

Barcode in

infrared beam

2.3Rotate

until price registers (“beep”)

2.4Verify

scanned price

=expected price)

2.3.1Price

does not register

2.4.1Price is

not correct

Go to 2.6 (FAIL)



Write the Test Script

• Script everything – even your greeting. Consistency is key!

• Start with your goals• Consider how learnable your task is

– Switch the order of tasks• What are you trying to observe?

– Be careful not to help them if they get stuck



Use various prototypes

• Prototyping: keep it simple• Don’t have enough budget? Consider paper

prototyping.



Facilitate a test

• Facilitating is an art• Professionals learn as an apprentices with little

formal practice• The importance of “Think aloud”• Dumas’ and Loring’s (10) Golden Rules:

– Participants are the experts; you are in charge.– Let the participants speak!– Be unbiased.

• Moderator role: all logistics, pacing, interaction



Plan for observers

Pros Cons

See the usability items under consideration

Can be distracting

Always surprised by what they see Can make participants feel uncomfortable

May build confidence in UX May latch on to single observation



Incorporate additional methods (card sorts, system usability scales)

• Card sorts• System Usability scales• Product reaction cards



Card sorts: results



System Usability Scale


© 2013 Teresa Washburn and John McGloon 19

System Usability Scale results

Aggregate usability score:

81



Product reaction cardsAccessible Desirable Gets in the way Patronizing Stressful

Appealing Easy to use Hard to use Personal Time-consuming

Attractive Efficient High quality Predictable Time-saving

Busy Empowering Inconsistent Relevant Too technical

Collaborative Exciting Intimidating Reliable Trustworthy

Complex Familiar Inviting Rigid Uncontrollable

Comprehensive Fast Motivating Simplistic Unconventional

Confusing Flexible Not valuable Slow Unpredictable

Connected Fresh Organized Sophisticated Usable

Consistent Frustrating Overbearing Stimulating Useful

Customizable Fun Overwhelming Straight Forward Valuable



Product reaction cards: results



Summarize a usability test: sample outline

1. Overview or executive summary2. Goals3. Methodology4. Participant Profile5. Tasks6. Findings / Recommendations7. Appendix(And the “question” of highlights.)



Summarize a usability test

• Audience and purpose• Budget and timing• Organizing

– By severity– By category– Content– By action required


© 2013 Teresa Washburn and John McGloon 25

Overview Screen: Usability Issues

Update Numbers

Some participants were unsure what the update numbers referred to. One thought they represented summary info of the maintenance alerts below. Others thought they referred to the number of devices in each of those states.

Severity

3 - Moderate© 2013 Teresa Washburn and John McGloon


Questions about anything we’ve covered:

• Write better screeners by linking personas• Prioritize with task sampling and selection• Write a test script • Use various prototypes (no prototyping will be done,

but examples will be provided)• Facilitate a test• Plan for observers• Incorporate additional methods (card sorts, system

usability scales)• Summarize a usability test



Further reading

• Moderating Usability Tests: Principles and Practices for Interacting (Interactive Technologies) by Joseph Dumas and Beth Loring.

• User and Task Analysis for Interface Design by JoAnn Hackos, and Janice Redish.

• A Practical Guide to Usability Testing by Joseph Dumas and Janice Redish.



Interested to learn more?

If you would like more information about usability testing, the user experience field, or how technical communication fits with those, feel free to provide your contact information and we’ll be in touch.

Thank you!


Design usability tests to make data-driven design decisions by Teresa Washburn and John McGloon

Design

Transcript of Design usability tests to make data-driven design decisions by Teresa Washburn and John McGloon