Seeing is believing: Using systematic observations in user experience research.
description
Transcript of Seeing is believing: Using systematic observations in user experience research.
![Page 1: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/1.jpg)
Seeing is believing:
Using systematic observations in user experience research.
March 11th, 2011
Bram van Mil [email protected] consultant
![Page 2: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/2.jpg)
![Page 3: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/3.jpg)
Noldus Information TechnologySystematic ObservationsMeasuring Behavior with The Observer XT Observation labs and other research tools
Agenda
![Page 4: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/4.jpg)
International developer of professional software for behavioral research
From a single software license up to a complete observation lab
Complete with installation, training and support
HQ located in Wageningen, The Netherlands
Founded in 1989, currently ≈
115 employees
Customers in 75 countries
Installed base Observer
software ≈ 4400 organizations
Who we areNoldus Information Technology, since 1989…
![Page 5: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/5.jpg)
![Page 6: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/6.jpg)
Why study behavior?
Develop new drugs to cure diseases Support medical diagnosis and therapy Support human resource management Improve educational systems Improve animal welfare Control insect pests Create more usable products Create safer work environments Optimize team performance Etc.
![Page 7: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/7.jpg)
Questionnaires
Observations
Ways to study behavior
![Page 8: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/8.jpg)
• Problem of meaning (interpretation of a question)
• Problem of omission (inadvertently leaving out our relevant information)
• Problem of memory
• Social desirability effect
• Question threat (questions may appear threatening)
• Interviewer characteristics (influence on participant)
• Gap between stated and actual behavior
Disadvantages of questionnaires
![Page 9: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/9.jpg)
Record who does what, where, when and with whom
The alternative: Systematic observations
![Page 10: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/10.jpg)
Objective
Accurate
Standardized
Quantify qualitative data
Advantages of systematic observations
![Page 11: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/11.jpg)
A human observer watches the behavior of thesubjects and writes down the observations on paper and uses a clock for time-information.
Disadvantages: Subjective measurements When writing, the observer cannot observe Very labor intensive No integration with other modalities
Measuring Behaviorthe pioneers way…
![Page 12: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/12.jpg)
Create a coding scheme before or during the observations and annotate behavior by a simple key press.
This will result in an event-log which contains time-stamped events that can be synchronized with multimodal data: Videos Physiological data Eye tracking data Etc.
Solution: The Observer XTfor structured observations
![Page 13: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/13.jpg)
How to execute systematic observations
Three steps
1. Setup
2. Observe
3. Analyze
![Page 14: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/14.jpg)
How to execute systematic observations
Step 1 – SetupObservation sources
![Page 15: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/15.jpg)
How to execute systematic observations
Step 1 – SetupCoding scheme
Subjects
Behaviors- Point events- State events
Modifiers
![Page 16: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/16.jpg)
How to execute systematic observations
Step 1 – SetupCoding scheme : Example
Time Subject, behavior, modifiers
09:35:12 Joe, run, fast, North
09:40:23 Mary, talk, Rose
09:40:24 Rose, smile
09:40:24 Rose, affect, 5
09:40:31 Mary, smile
![Page 17: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/17.jpg)
How to execute systematic observations
Step 1 – SetupCoding scheme : Mutually exclusive behaviors
Mutually exclusive: one behavior ends the previous
![Page 18: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/18.jpg)
How to execute systematic observations
Step 2 – Observe
Live scoring: With or without videos
Offline scoring: From digital video files
![Page 19: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/19.jpg)
How to execute systematic observations
Step 3 – Analyze Data selection
Choose the data you want to analyze by filtering and nesting your data
This is done by data profiles: In order to save different criteria for you data
selection and data analysis
![Page 20: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/20.jpg)
How to execute systematic observations
Step 3 – Analyze Visualization of your data
![Page 21: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/21.jpg)
How to execute systematic observations
Step 3 – Analyze Descriptive statistics
Advanced analysis - Lag-sequential analyses - Reliability analysis
Statistical analysis - Export to Excel, SPSS or as ASCI file
Export highlight videos
![Page 22: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/22.jpg)
How to execute systematic observations
Step 3 – Analyze: Numerical analysis on external data
![Page 23: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/23.jpg)
Other toolsThat can be used in behavioral research
![Page 24: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/24.jpg)
Neutral Happy Sad Angry Surprised Scare Disgusted
Also: gender, age, ethnicity, glasses and facial hair
FaceReaderDetect facial expressions
![Page 25: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/25.jpg)
Eye Tracking SystemsSee what your participants are looking at
Computer monitor / Standalone / Head mounted Gaze tracks, heat maps, fixations and areas of interest
![Page 26: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/26.jpg)
Physiological dataDetect mental load, physical load and emotional states
ECG – Electro Cardio Gram EMG – Electro Myo Gram EEG – Electroencephalogram GSR – Galvanic Skin Response
![Page 27: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/27.jpg)
LabsComplete setups for behavioral research
![Page 28: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/28.jpg)
Portable Usability Lab
![Page 29: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/29.jpg)
Portable Observation Lab
![Page 30: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/30.jpg)
Custom Mobile Observation Labs
![Page 31: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/31.jpg)
Stationary Observation Lab
![Page 32: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/32.jpg)
Game Experience LabGhent University
Two room Usability Lab for User Experience research in gaming on consoles and pc’s
![Page 33: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/33.jpg)
Game Experience LabGhent University
![Page 34: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/34.jpg)
Game Experience LabGhent University
![Page 35: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/35.jpg)
Different facilities under 1 roof
– Observational-, sensory- and mood- labs
Observe eating behavior
– Introduce better suited products
Restaurant of the FutureWageningen University
![Page 36: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/36.jpg)
Restaurant of the FutureWageningen University
![Page 37: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/37.jpg)
Service, training and supportWe are there when you need us
On-site installation and training Behavioral scientists Hardware engineers
Excellent technical support Helpdesk Online knowledge base
Other Service contracts Measuring Behavior conference
![Page 38: Seeing is believing: Using systematic observations in user experience research.](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163e5550346895dd5452d/html5/thumbnails/38.jpg)
Bram van Mil [email protected] consultant