Collaborative Research
with Erika Hall (@mulegirl)
UXLX
Hello!
I have a question...
Do you enjoy being right?
You are correct!
YESSS!
p0wned!
No.
?
>? !
Dogma
Flickr/Chris Voll
Ego!
Why?
People!
Collaboration!
TextAgenda
TextAgenda
9–10:30
10:30–11
11–12:30
Introduction.
The research process
Questions and activities
Understanding your organization
Break
User research
Analysis
Models and reports
Finale
Barriers
Overcoming Objections
Laziness
Laziness
Fear
Laziness
Fear
Lunch
Laziness
Fear
Lunch
Following
Laziness
Fear
Lunch
Following
Losing Control
Laziness
Fear
Lunch
Following
Losing Control
Sharing
Shared Understanding
We don’t havethe time.
We don’t havethe money.
We don’t have the expertise.
We’re already A/B testing
Everyone wants better products, faster.
No one wants to read a report.
What is your experience?
Research + Collaboration
A design project isa series of decisions.
Data doesn’t change minds.
Whatis
What ought to be
Design-Led
Research-Led
ExpertMindset
ParticipatoryMindset
Users seen as subjects Users seen as partners
Design-ledwith
expert mindset
Design-ledwith
participatory mindset
Research-ledwith expert mindset
Research-ledwith participatory
mindset
Dubberly Design Office
Goal Driven
Skeptical Mindset
Increase chance of successReduce risk
Willing to question the value of any approach
Team + Goal + Reality = Good
One Simple
Process
Form Questions
AnalyzeData
GatherData
Form Questions
AnalyzeData
Think Critically
Form Questions
AnalyzeData
Observe
Form Questions
AnalyzeData
Interview
Form Questions
AnalyzeData
Read
Form Questions
AnalyzeData
Read
Experiment
Interview
Observe
Think
PersonalView
PersonalView
PersonalView
SharedReality
Researchis a Craft
Form Questions
AnalyzeData
GatherData
Questions determine results.
Questions give research meaning.
Research high-priority questions.
Good Questions
SpecificActionablePractical
A Bad Question
“What do people think about pets?”
A Better Question
“How do single urban adults choose and acquire a pet?”
A Bad Question
“What do people do around here all day?”
A Better Question
“How do editors and designers work together?”
The Best Question
The unknown with the most risk.
Bias
Bias: Something that causes an influence or prejudice
Confirmation Bias:
You selectively weight the information that confirms what you already believe.
Sampling Bias:
Your sample of research subjects isn’t sufficiently representative.
Interviewer Bias:
You insert your opinion into interviews.
Social Desirability Bias
People don’t say the true things that they worry will make them look bad.
Ease
Clear Display
Related
Primed Idea
Good Mood
Feels True
Feels Familiar
Feels Good
Feels Effortless
Daniel Kahneman
Feeling confident?It’s not a good sign.
You might have a bad case ofDunning-Kruger.
Critical Thinking
Critical Thinking
DisciplinedSelf-correctingClearLogical
Uncritical Thinking
“I hate yellow, so a yellow website won’t succeed.”
Critical Thinking
“I hate yellow, but based on the evidence, it might work for our audience.”
Critical Thinking
“I don’t know.”
Activities!
Form Questions
AnalyzeData
GatherData
Questions About
Users
ProductOrg
Competition
InterviewsInterviews
UsabilityTesting
A/BTesting
ContextualInquiry
LiteratureReview
SWOTAnalysis
BrandAudit
Usability Testing
CompetitiveAnalysis
HeuristicAnalysis
Descriptive
Evaluative
Evaluative
Evaluative
Analytic
Analytic
Generative
Descriptive
ResearchActivity
TopicPurpose
Time
Money
Phone InterviewsWhat do we
need to know about?
What kind of decision will it
inform?
How long do we have?
What is our budget?
Contextual Inquiry
In-Person Interviews
Usability Testing
Competitive Analysis
Why not just make a
prototype?
20
If we only test bottle openers, we may never realize customers prefer screw-top bottles.
– Victor Lombardi, Why We Fail
Topics
Organizational Research
Stakeholders
Stakeholders
Executives
Sales People
Customer Service
Editors
Production Team
Organizational research helps you with:
Requirements
Politics
Workflow
Capabilities
Goodwill
Requirements
What are the top business priorities for this project/product?
Politics
What does success mean to the individual stakeholders?
Workflow
Do we have to change how people are working together to be successful?
Workflow
How do we have to change how people are working together to be successful?
Workflow
How can we possibly change how people are working together?
Capabilities
What are the strengths and weaknesses of our team?
Capabilities
Where is the internal expertise?
Goodwill
How can this project make your job easier (or harder)?
Get them alone
Basic Stakeholder Questions
What is your title? How long have you been in this role?
What are your essential duties and responsibilities?
What does a typical day look like?
Who are the people you work most closely with? How is that going?
What does success mean from your perspective, what will have changed for the better once this project is complete?
Do you have any concerns about this project?
What do you think the greatest challenges to success are? Internal and external?
For each stakeholder, note the following:
What’s their general attitude toward this project?
What’s the goal as they describe it?
To what extent are this person’s incentives aligned with the project’s success?
How much and what type of influence do they have?
Who else do they communicate with on a regular basis?
To what extent does this stakeholder need to participate throughout the project, and in which role?
Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?
Stakeholder power moves
“Why are you asking me this?”
“I don’t understand that question. It doesn’t make any sense.”
“I don’t feel comfortable talking to you about that.”
“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”
“How much more time is this going to take?”
Practice!
10 minutes practice. Find a partner. Take turns.
What is your title? How long have you had this job?
What are your essential duties and responsibilities?
What is a typical day like?
Who are the people you work most closely with? How is that going?
What do you think the greatest challenges to your success are? Internal and external?
Empathy
Break!
To Review
Team + Goal + Shared Reality = Good
Research is a simple process you can apply to however you work. You shouldn’t be dogmatic.
Even though this sounds obvious, some people will resist this because questions can feel threatening.
Facts will not change the minds of people who are threatened.
You need to appeal to what you know is important to them, and fit your facts into their story.
So, understanding what is important to your stakeholders is necessary for design and research to succeed.
UserResearch
Photo: Flickr/theloushe
Ethnography
How to do bad user research:Ask people what they want.
How to do bad user research:Ask people what they like.
Never ask users what they want or like.
The Four Ds of Design Ethnography
Deep DiveDaily Life
Data AnalysisDrama
“...true ethnography reveals not just what people say they do, but what they actually do.”
–PARC
Photo: Flickr/lintmachine
The Art ofThe Interview
Interviewing is not talking.
Interviewing is listening.
Good Interviewers:
Know Your Question
Warm Up
Shut Up
Interview Structure:
Introduction
Body
Conclusion
Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions
Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist
Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time
You are the hostYou are the student
Out of your comfort zone,and into theirs.
Interview ChecklistCreate a welcoming atmosphere to make participants feel at ease.
Always listen more than you speak.
Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.
Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.
Avoid leading questions and closed yes/no questions. Ask follow-up questions.
Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.
Also note the exact phrases and vocabulary that participants use.
Look forGoals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment
Roles
Interviewer
Notetaker
Observer
Practice!
Interview Scenario
You work for an e-commerce site that wants to develop a new service to help people give gifts.
The goal of the research is to identify unmet needs people might have with regard to giving gifts.
Interview Practice
Break into groups of 3-4 people
1 interviewee, 1 interviewer , 1 notetaker, 1 observer (optional),
Switch in 15 minutes
2 rounds
Look forGoals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment
How did that go?
How about a focus group?
14
“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.”
–Robert K. Merton, Sociologist,invented focus groups
Everybody Lies
Competitive Research
How else might your target customer solve
the same problem?
Competitive Review
What do they say they offer?
Who is their customer? How is this the same or different from your target audience or users?
What are the key differentiators—the factors that make them uniquely valuable to their target market, if any?
How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve?
What do you notice that they’re doing particularly well or badly?
Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?
Your target customershave to love youmore than they
hate change.
(Usability) Testing
A good research activity:
•Answers a key question
•Addresses identified assumptions
•Informs specific decisions
•Involves your team
•Fits your level of expertise
•Fits your schedule and budget
•Fundamentally research is a simple process
•There are many activities and definitions
•No pressure!
•Select the methods that inform decisions
•Begin by understanding your organization
•Never ask what people like
•People are lazy, forgetful creatures of habit
•Keep each other honest
•Practice and learn
Research and Collaboration
Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be.
Done right, research and working collaboratively reinforce each other through a shared understanding of reality.
Start with your goal in mind, not with any process or buzzword.
Asking questions and cutting across traditional roles can both be threatening to the established order.
Commit to clear communication and critical thinking.
Research questions follow from goals, assumptions, and risk.
Always have a framework and a plan.
Creating Meaning From Data
1. Compile data2. Analyze3. Identify Insights4. Create Model
Basic Analysis
Closely review the notes.
Look for interesting behaviors, emotions, actions, and verbatim quotes.
Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back).
Group the notes.
Watch the patterns emerge.
Rearrange the notes as you continue to assess the patterns.
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Collaborates on purchases
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Collaborates on purchases
Uses several devices
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Collaborates on purchases
Uses several devices
Needs affirmation
Observation
Observation
Observation
Observation
Observation
Observation
Observation
Ground rules
Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal.
Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data.
Clearly differentiate observations from interpretations (what happened versus what it means).
No specific solutions until after you’ve gone through insights and principles. Solutions come next.
Practice!
20 minutes analysis.
Break into groups of 6-8 people
Each group work together to fill out one diagram with the strongest patterns.
Negotiate and advocate for your perspective.
Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment
20 minutes analysis.
Break into groups of 6-8 people
Each group work together to fill out one diagram with the strongest patterns.
Negotiate and advocate for your perspective.
How did that go?
Models
Reporting
You are collaborating withyour future selves.
Research ReportStudy Title
Date Completed
Research Goal
Activities
Related Decisions
Key Insights
Supporting Observations
Recommended Actions
Questions for Further Study
A useful report supports
Clear goals
Shared values
Access to information
Clear decision-making
You decide if it’s important for the report to be
Informing?
Inspiring?
Focusing?
Remembering?
Recording?
Deciding?
Finale
In summary
Research creates a shared understanding of reality.
Asking questions is uncomfortable. Embrace that feeling.
A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.
Clear goals and good questions are required.
Choose only the research activities that answer real questions and inform your top priority design and development decisions.
Practice! Observe and listen every day.
Document! Report! Share! It’s easy to lose what you learn.
Any questions?
Additional sources:Designing Together by Dan A. Brownhttp://www.designingtogetherbook.com/
LeanUX by Jeff Gothelfhttp://www.leanuxbook.com/
Remote Research by Nate Bolt & Tony Tony Tulathimuttehttp://rosenfeldmedia.com/books/remote-research/
Interviewing Users by Steve Portigalhttp://rosenfeldmedia.com/books/interviewing-users/
Google Ventures Library | Designhttp://www.gv.com/library/design/
Pacific Standard Magazine http://www.psmag.com/
Helsinki Design Lab (closed, but excellent publications still available)http://www.helsinkidesignlab.org/pages/publications
Brief books for people who make websites No.
9
JUST ENOUGHRESEARCH
Erika Hall
You might enjoy the book.
www.abookapart.comFor 15% off, use code: UXLXJER14
Top Related