Languages
Pages
Legal
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013
Covert Naturalistic Observation
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013
Covert Naturalistic Observation - Jim Ross 4
User research is unnatural.
Covert Naturalistic Observation - Jim Ross 5
User research is unnatural.
Source: Flickr – Thomas Link Source: Flickr – Thomas Link
Covert Naturalistic Observation - Jim Ross 6
User research is unnatural.
Covert Naturalistic Observation - Jim Ross 7
We get informed consent from participants.
Covert Naturalistic Observation - Jim Ross 8
But knowing that you’re being observed affects behavior.
Source: Flickr – Blue Oxen
Covert Naturalistic Observation - Jim Ross 10
Covert naturalistic observation
Source: Flickr – Sean Hobson
Covert Naturalistic Observation - Jim Ross 11
We usually use overt naturalistic observation.
Covert Naturalistic Observation - Jim Ross 12
It’s rare that we use covert naturalistic observation.
Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross 13
Who uses covert naturalistic observation?
Covert Naturalistic Observation - Jim Ross 14
Who uses covert naturalistic observation?
Covert Naturalistic Observation - Jim Ross 15
Who uses covert naturalistic observation?
Covert naturalistic observation
Covert Naturalistic Observation - Jim Ross 17
It allows you to observe realistic, natural behavior.
Source: Flickr – David Hillowitz
Covert Naturalistic Observation - Jim Ross 18
Trying to remain covert makes observation more difficult.
Source: Flickr – Lars Plougmann
Covert Naturalistic Observation - Jim Ross 19
You can’t study interfaces or close up interactions.
Source: Flickr – Walter Lim
Covert Naturalistic Observation - Jim Ross 20
You can’t ask questions, so you rely on observation only.
Source: Flickr – Danielle Scott
Covert Naturalistic Observation - Jim Ross 21
Ethical concerns about consent, privacy, and deception
Source: Flickr – dustpuppy
Covert naturalistic observation
Covert Naturalistic Observation - Jim Ross 23
Studying behavior and interactions in a public place
Source: Flickr – dustpuppy Source: Flickr – Drew XXX
Covert naturalistic observation
Covert Naturalistic Observation - Jim Ross 25
1. Plan what you want to observe.
Covert Naturalistic Observation - Jim Ross 26
2. Gather intelligence – do some research on:
• The domain
• User characteristics
• Tasks and behaviors
• Existing research
Covert Naturalistic Observation - Jim Ross 27
3. Scope out the location ahead of time: layout, people, activity level.
Source: Flickr – Ben Salter
Covert Naturalistic Observation - Jim Ross 28
4. Consider notifying management and security.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross 29
5. Observe with partners.
Source: Flickr – Carey Akin
Covert Naturalistic Observation - Jim Ross 30
6. Observe over several shorter sessions to see:
• More people
• Different situations
• Different time periods
Covert Naturalistic Observation - Jim Ross 31
6. Observe over several shorter sessions to avoid:
• Fatigue
• Information overload
• “Getting caught”
Covert Naturalistic Observation - Jim Ross 32
7. Blend in – dress like everyone else.
Source: Flickr – Michael Ocampo
Covert Naturalistic Observation - Jim Ross 33
7. Blend in – do what others are doing.
Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross 34
7. Blend in – use a mobile device.
Source: Flickr – Elvert Barnes Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross 35
8. Don’t take obvious notes.
Source: Flickr – Geek Calendar
Covert Naturalistic Observation - Jim Ross 36
8. Don’t take obvious notes – use a phone or tablet.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross 37
9. Treat recording as secondary – take photos or video last.
Covert Naturalistic Observation - Jim Ross 38
9. Treat recording as secondary – use a phone or tablet.
Source: Flickr – Jesus Leon
Covert Naturalistic Observation - Jim Ross 39
9. Treat recording as secondary – don’t record audio.
Covert Naturalistic Observation - Jim Ross 40
10. Debrief after each observation session.
Source: Flickr – Sean MacEntee
Covert Naturalistic Observation - Jim Ross 41
10. Debrief after each observation session.
Covert Naturalistic Observation - Jim Ross 42
11. Do it ethically – observe in public places.
Source: Flickr – Dominic Alves
Covert Naturalistic Observation - Jim Ross 43
11. Do it ethically – focus on groups not individuals.
Source: Flickr – Alfonso Pierantonio
Covert Naturalistic Observation - Jim Ross 44
11. Do it ethically – protect privacy.
Source: Flickr – David Goehring
Covert Naturalistic Observation - Jim Ross 45
11. Do it ethically – don’t deceive.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross 46
12. Combine it with other methods.
Covert naturalistic observation
Overt and covert methods each have advantages and disadvantages.
48 Covert Naturalistic Observation - Jim Ross
With understanding of the advantages and disadvantages, we can use overt and covert methods as appropriate to best understand the user experience.
49 Covert Naturalistic Observation - Jim Ross
Covert naturalistic observation
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013
Top Related