The agency's guide to effective user research
-
Upload
usertesting -
Category
Design
-
view
153 -
download
1
Transcript of The agency's guide to effective user research
#UTwebinar @UserTesting
Aiden Bordner Co-founder and Principal Designer
Parade
The agency’s guide to effective user research
The Agency’s Guide to Effective User Research #UTWebinar
Who is this for?
•Large Agencies •Small Agencies •Anyone who works with Stakeholders
The Agency’s Guide to Effective User Research #UTWebinar
Why do agencies struggle?
• Large Agencies: • Traditionally, testing was to be avoided • Hierarchical nature, focus on client relationship • Perception of high cost • Fear of “bad results”
Issues largely to do with Culture and Business Model
The Agency’s Guide to Effective User Research #UTWebinar
Why do agencies struggle?
• Small Agencies: • No staff on hand to focus on research • Project-basis and of fixed-fee scopes • No experience doing research before
Issues largely to do with Culture and Business Model
The Agency’s Guide to Effective User Research #UTWebinar
How do we improve?
1. Make Testing a Rule 2. Set Expectations Early, Often 3. Develop a Broad Toolkit to Right-Size Research 4. Reduce Costs by Learning to Moderate Yourself 5. Include the Client in Testing 6. Proactively Manage Results
Six tips for doing more effective user research and overcoming cultural and business impediments.
The Agency’s Guide to Effective User Research #UTWebinar
1. Make Testing a Rule
• Best way to ensure testing is to change culture • Don’t let yourself take it out of proposals • Don’t let the client take it out either • Cite monetary or performance sales examples
The Agency’s Guide to Effective User Research #UTWebinar
1. Make Testing a Rule
• How to schedule • Plan on testing for every ~3-4 weeks of design. • Testing should take 1-2 weeks depending on
methodology • You can keep designing while you’re testing • Include time for revisions
The Agency’s Guide to Effective User Research #UTWebinar
2. Set Expectations Early, Often
• Work to diffuse feeling that testing is a “Big Deal” • Not market research • Qual vs. quant • Formalize expectations in a deliverable
• (e.g. Research Plan)
The Agency’s Guide to Effective User Research #UTWebinar
Benefit Cosmetics • Brows Digital Experience • Outside Test Research Plan Page / 1 4
BENEFIT BROWS DIGITAL EXPERIENCE
Outside Research Planv1.0 December 8, 2015
Research Plan
• 1-3 page document • Sections:
• Overview
• Methodology
• Stimulus
• Area of Focus
• Areas of Non-Focus
• Known Limitations
• Outcomes/Reporting
• Schedule
• Share at beginning of each research effort
Solidifies Plan & Sets Expectations
The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Learn to develop a broad kit of tools
The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Recruiting:
• Services • Live Recruiting • Friends & Family • Building Occupants • Client Employees • Intercepts • Craigslist • Pro-Tip: Look to Client for Incentives
Develop a Broad Kit of Tools
The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Usability Testing:
• DIY vs. Full-Service • In-Person, Remote • Moderated, Unmoderated • Free Exploration, Task Completion, Natural Task • “Hallway” Testing
Develop a Broad Kit of Tools
The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• User Research:
• Great way to focus before designing • Simple Observation • Interviews • Card Sorts, Tree Testing
Develop a Broad Kit of Tools
The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Learn to develop a broad kit of tools • Discuss options, recommendations with client • Keep the recruit small • Downsize vs. eliminating
The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating internally saves time and money • Two key outputs:
• Discussion Guide • Moderation
The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Discussion Guide:
• Script For Moderation • Sections:
• Orientation/Ease-Setting • Recording Permission • “Think Aloud” Conditioning • Self-Directed Exploration or Natural Task Repeat • Task Completion • Open-Ended Questions • Thank You/Incentive Information
Introduction (2 minutes)TOPIC MODERATOR
Introduction So, thanks again for participating. My name is _________, I’m going to be guiding you though this experiment today. This is ________, who’ll just be taking notes.
Before we get started I would like to tell you a little bit about what we are doing today. We’re collecting feedback on some work in progress designs for a future Benefit website. This website will be about eyebrows—both in general and about specific products for eyebrows, some of which you may be familiar with and some of which may be new. We’ll be testing just a [mobile or desktop] version of the website today, so in a few minutes I’m going to [hand you this mobile device or ask you to use this laptop] which is set up to record your actions and where you tap on the screen so we can review it later.
Makes sense?
Orientation / Relaxation
Your job is really easy, you just have to be yourself and act as you naturally would. Our goal today is to get honest feedback, so feel free to say whatever’s on your mind—you won’t offend us. As you interact with the site, you should know this is just a prototype—a work in progress—so some things may not be fully functioning or complete. If something doesn’t work, just tell me what you expected to happen instead.
[Mobile only: You should also know there’s one point where we may need to change apps for the test. If that’s necessary, I’ll let you know when, and you can just hand to phone to me and I’ll make the switch before we continue. We’ll have to switch back when we’re done, but it will only take a second.] Okay? Great!
Start Recording So I’m just going to start the recording on this device and we’ll be almost ready to go.
[Hand device to participant or position participant at laptop] Okay, and just for the record, could you please look at the screen and say your name for us.
… Okay great!
Brows Digital Experience • Hallway Test Moderator Guide Page / 2 6
The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating:
• Stop Being the Designer • Above All: Avoid Leading Questions • Answer Questions with More Questions • Give the Subject Time; Never Interrupt • When in Doubt, Don’t Talk • Don’t Get Frustrated • Be Encouraging and Thankful
The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating internally saves time and money • Two key outputs:
• Discussion Guide • Moderation
• Don’t worry: trust your team
The Agency’s Guide to Effective User Research #UTWebinar
5. Include the Client
• Important to give clients real-time visibility • Use technology to create a virtual lab • Place someone with the client, if possible • Schedule discussion breaks • Provide recordings • Do a Tech Rehearsal
The Agency’s Guide to Effective User Research #UTWebinar
6. Manage Results Proactively
• Proactively filter the “noise” • Focus only on the trends • Explain behavior vs. comprehension • Make design recommendations proactively • Dispel non-issues proactively, too • Keep it lean • Don’t bludgeon your client with research
The Agency’s Guide to Effective User Research #UTWebinar
Recap
1. Make Testing a Rule 2. Set Expectations Early, Often 3. Develop a Broad Toolkit to Right-Size Research 4. Reduce Costs by Learning to Moderate Yourself 5. Include the Client in Testing 6. Proactively Manage Results
Six tips for doing more effective user research and overcoming cultural and business impediments.
The Agency’s Guide to Effective User Research #UTWebinar
Our Approach
1. Plan to do a study for every 3-4 weeks of design
2. Do upfront user research for any project involving unknown user types, special interactions
3. Build research into the schedule at time of proposal 4. Three Deliverables:
• Research Plan
• Discussion Guide (if necessary)
• Report
Parade’s process for research scheduling
The Agency’s Guide to Effective User Research #UTWebinar
Our Approach
5. Should be able to accomplish a moderated, one-day study in 7 working days:
• Day 1: Review, adjust and approve research plan
• Day 2: Review moderation Guide
• Day 3: Finalize guide
• Day 5: Testing
• Day 7: Present findings
Parade’s process for research scheduling
The Agency’s Guide to Effective User Research #UTWebinar
Questions?
• Learn more about Parade and Aiden’s team at • parade-sf.com
• Questions about UserTesting? • Email [email protected]