Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth Ellison
52
The researcher’s blind spot: 7 cognitive biases we shouldn’t ignore in research UX Singapore 2016 Ruth Ellison, Principal User Researcher, PwC’s Experience Centre @RuthEllison
-
Upload
ux-singapore -
Category
Design
-
view
688 -
download
2
Transcript of Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth Ellison
The researcher’s blind spot: 7 cognitive biases we shouldn’t ignore in research
UX Singapore 2016 Ruth Ellison, Principal User Researcher, PwC’s Experience Centre @RuthEllison
@RuthEllison from PwC’s Digital Services
Hi :)
2
@RuthEllison
Free!
$10
$7
$20
@RuthEllison
$1
$10
$8
$20
5
Kiasu
Mr. Kiasu by Johnny Lau (Source: http://www.nlb.gov.sg/sure/wp-content/uploads/2013/10/Mr-Kiasu.jpg)
@RuthEllison from PwC’s Digital Services
The Zero Price Effect is a Cognitive Bias
@RuthEllison
What is a cognitive bias? Mental shortcut
@RuthEllison 8
We are all human Photo available under a CC by 2.0 licence https://www.flickr.com/photos/yellowbookltd/14907364551/
@RuthEllison Image source: http://www.businessinsider.com.au/what-new-coke-looked-like-in-1985-2013-3
@RuthEllison
Applying it to design research
Preparing Running
Analysing
@RuthEllison from PwC’s Digital Services
Preparing for research
@RuthEllison
@RuthEllison from PwC’s Digital Services
Selection bias
@RuthEllison 14
@RuthEllison from PwC’s Digital Services
Multiple channels - mix of recruitment companies, social media, trusted networks
Avoid professional respondents
Careful screeners
Behavioural based recruitment
Dealing
with it….
@RuthEllison from PwC’s Digital Services
Running research sessions
Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153
@RuthEllison from PwC’s Digital Services
Observer-Expectancy effect
@RuthEllison 18
Source: https://medium.com/@userfocus/the-7-deadly-sins-of-user-research-22857c5a971b#.xb38ukdtr
@RuthEllison
The way we frame our questions matter
@RuthEllison
Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589.
McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html
About how fast were the cars going when they smashed into each
other?
@RuthEllison
About how fast were the cars going when they collided into each other?
About how fast were the cars going when they bumped into each other?
About how fast were the cars going when they contacted into each other?
About how fast were the cars going when they smashed into each other?
@RuthEllison from PwC’s Digital Services
Triangulate research
Use observational methods
Keeping positive-neutral body language, watch the tone of your voice
Avoid leading questions
Dealing
with it….
@RuthEllison 23
I need a volunteer please
24
2 4 8
PwC’s Digital Services
What’s going on?
@RuthEllison from PwC’s Digital Services
Confirmation bias
@RuthEllison 27
28
Take the opposing view
29
Don’t just confirm your hypothesis, see if you can prove it wrong
Analysis of Competing Hypotheses (ACH)
30
Use open ended questions
Some people think that soft drinks are bad for you. What do you think?
What’s your opinion about soft drinks?
@RuthEllison from PwC’s Digital Services
List assumptions
Be skeptical, especially if everyone agrees with you
Remain open
Consider all evidence equally
Multiple user researchers
Leave your ego by the door.
Dealing
with it….
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
Groupthink & the Bandwagon effect
@RuthEllison 33
@RuthEllison from PwC’s Digital Services
Triangulate with other research methods (e.g. observational)
Use a mixture of individual and group exercises
Avoid stating preferences and expectations at start
Give someone the devil’s advocate role to question assumptions
Dealing
with it….
@RuthEllison from PwC’s Digital Services
Anchoring bias
@RuthEllison 36
37 https://www.flickr.com/photos/naotakem/17105160150/
@RuthEllison from PwC’s Digital Services
Consider the order of questions and designs carefully
Open ended questions
Alternate order in which participants are shown concept or design versions
Show version A first 1 3 5
2 4 6 Show version B first
Dealing
with it….
@RuthEllison from PwC’s Digital Services
Analysing
@RuthEllison 40
@RuthEllison
Apophenia
41 Source: https://au.pinterest.com/pin/249738741807276910/
@RuthEllison from PwC’s Digital Services
Clustering illusion & reporting bias
@RuthEllison from PwC’s Digital Services
Sample sizes – it’s about the WHY
Consider evidence equally – not just the ones that confirm your belief/assumption
Collaborative analysis sessions
Strive for objectivity
Dealing
with it….
@RuthEllison 44
How vulnerable are you to these biases?
@RuthEllison from PwC’s Digital Services
What percentage of the population will be more biased
than the average person?
@RuthEllison from PwC’s Digital Services
Are you more biased than the average person?
@RuthEllison from PwC’s Digital Services
Blind spot bias
@RuthEllison from PwC’s Digital Services
Listen with an open mind
Become more rational, but less rationalising
Continuous learning
48
Always assess your method, your analysis and yourself for
bias
@RuthEllison 49
We are all human
@RuthEllison Image source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions_1954.JPG
Thank you :)
Say hello @RuthEllison
@RuthEllison
Further reading
List of cognitive biases You are not so smart: a celebration of self delusion
9 Biases In Usability Testing
52
