Eye Tracking the UX of Mobile: What You Need to Know

Eye tracking the UX of mobile: What you need to know

Jen Romano Bergstrom UX Researcher Facebook @romanocog

1

Jon Strohl UX Researcher Fors Marsh Group @jonstrohl

March 23, 2015 ModevUX | Arlington VA

We will be answering these questions!

2

Ø  Why is mobile important?

Ø  What makes mobile different?

Ø  Do I need to test my design? •  How do I get started? •  What are the benefits?

Ø  Do I need to include eye tracking? •  How do I get started? •  What are the benefits? •  Can you show me how it’s done? •  Are there any examples?

The rise of mobile

3

@romanocog @jonstrohl #modevux

Mobile internet usage continues to increase

4

Mobile retail usage increases

5


App usage continues to increase

6

7

What we design for…and reality @romanocog @jonstrohl

#modevux

Krug, S. (2000) Don’t Make Me Think. Pearson Education *NEW 3rd Edition (2014)*

@skrug

8

8

Ø  Controlled environment

Ø  All par>cipants have the same experience

Ø  Record and communicate from control room

Ø  Observers watch from control room and provide addi>onal probes (via moderator) in real >me

Ø  Incorporate physiological measures (e.g., eye tracking, EDA)

Ø  No travel costs

LABORATORY REMOTE IN THE FIELD Ø  Par>cipants tend to be

more comfortable in their natural environments

Ø  Recruit hard-‐to-‐reach popula>ons (e.g., children, doctors)

Ø  Moderator travels to various loca>ons

Ø  Bring equipment (e.g., eye tracker)

Ø  Natural observa>ons

Ø  Par>cipants in their natural environments (e.g., home, work)

Ø  Use video chat (moderated sessions) or online programs (unmoderated)

Ø  Conduct many sessions quickly

Ø  Recruit par>cipants in many loca>ons (e.g., states, countries)

> FMG Capabilities

Where to test

9

When to test

Iterative Process Throughout Development


10

Research goals and

plan Form

hypotheses Will eye

tracking add value?

Design the study

Planning the test @romanocog @jonstrohl

#modevux

11

Who is involved

SATQ Items

Other materials

Debriefing

Reports

.

. .

.

. .

.

“Make it a spectator sport” @skrug


Project Managers

UX Designers

Executives

Engineers

Researchers

User Experience (UX) Defined

12


Adapted from Morville, P. Semantic Studios. http://semanticstudios.com/user_experience_design/

A combination of measures are used in UX research.


Using multiple measures allows researchers to capture the most detailed and complete picture of the user experience.

14

OBSERVATIONAL First click accuracy

Task accuracy Time on page/task

Selection/click behavior Conversion rate

SELF-REPORT Satisfaction ratings

Difficulty ratings Think-aloud protocol Debriefing interview

UX Measures @romanocog @jonstrohl

#modevux

15 http://blog.thomsonreuters.com/wp-content/uploads/2013/01/smartphones1.jpg

Lots of sizes to consider… @romanocog @jonstrohl

#modevux

16 http://armodilo.com/support/supported-tablets/

Don’t forget about tablets! @romanocog @jonstrohl

#modevux

…our thumbs are only so big

17 http://gizmodo.com/how-to-design-for-thumbs-in-the-era-of-huge-screens-1636955848

…and people use them differently.

18


Hoober (2013). UX Matters. http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php

Eye tracking and attention

19


@agabojko

Bojko, Aga (2013). Designing the User Experience. Rosenfeld Media

User Experience (UX) Defined

20

Eye tracking helps to

explain how usable,

engaging, and desirable a product is.


21

OBSERVATIONAL First click accuracy

Task accuracy Time on page/task

Selection/click behavior Conversion rate

PHYSIOLOGICAL Eye movements

Electrodermal activity (EDA) Pupil dilation

UX Measures @romanocog @jonstrohl

#modevux

SELF-REPORT Satisfaction ratings

Difficulty ratings Think-aloud protocol Debriefing interview

User Experience (UX) Measures •  Self-report metrics tell us why participants focus on

certain site aspects.

22

•  Eye tracking tells us what, how long, and how often participants focus on design elements.

•  The combination of self-report, observational, and implicit data allows us to better understand perceptions and behavior.

•  Observational metrics tell us how participants navigate and interact.

We do not use eye tracking in isolation.


23

Eye Tracking Considerations @romanocog @jonstrohl

#modevux

@agabojko

Bojko, Aga (2013). Designing the User Experience. Rosenfeld Media

User Experience Testing + Eye Tracking

24

UX Testing UX Testing with Eye Tracking

Data while completing tasks

Mouse movements, clicks, comments

Scan paths, dwell time, first fixation, re-fixations. Eye-tracking helps determine what led to these behaviors and comments

Findings Usability problems are detected

A more thorough understanding of usability issues and the sources of these issues

Recommendations Quality recommendations based on available data

More insightful and precise recommendations


Eye-Tracking Metrics ENGAGEMENT Number of fixations

Total dwell time Percentage of time on

an area

FINDABILITY Time to first fixation Number of fixations prior to first fixation

PROCESSING Fixation

durations

WORKLOAD/EXCITEMENT Pupil dilation

PROCESSING ORDER

Gaze path

COMPREHENSION Repeat fixations

25

26

Data Visualization– Heat & Opacity Maps @romanocog @jonstrohl

#modevux

27

Data Visualization– Gaze Plots @romanocog @jonstrohl

#modevux

28

Modern eye tracking @romanocog @dcaapor

Eye Tracking in the Past

29

Modern eye tracking

Modern Eye Tracking

Mobile Device Stand

30

Glasses

31

Shopping Research

32

Driving Research

33

Design Evaluation Research

34

Moderator & Participant

35 Slide from: Alayoubi, L. & Romano Bergstrom, J. (2014). Paying for Education: User-Centered Research on the Paying for College Website and Tools. Presentation at edUi, Richmond, VA, Sept 2014.

Example Methodology – Formative Evaluation

Eye Tracker

Participants: •  N = 9 •  Mix of gender, ethnicity, income. •  Formative evaluation with a single

web prototype.

User Experience Testing session: •  Participants read a description of

the study. •  The moderator gave instructions

and calibrated the eye tracker. •  Participants completed a set of

tasks provided individually by the moderator.

•  End-of-session satisfaction questionnaire.

•  Debriefing interview.

36

Formative Studies Ø  Are <10 participants appropriate?

•  Very often. Typical for finding large usability problems. •  Problem occurrence is 25% or higher. •  Problem discovery goal is 75% or less.

Ø  Are >20 participants appropriate? •  Not as common, but sometimes used when testing more mature

applications or trying to find smaller, less frequent issues. •  Problem occurrence is 10% or higher. •  Problem discovery goal is 90%.

Ø  If eye tracking, over recruit by ~20% 1.Calculations derived from:

Sauro, J., & Lewis, J. R. (2012). Quantifying the user experience: Practical statistics for user research. Elsevier. Bojko, A. (2013). Eye Tracking the User Experience. Rosenfield Media.

Determining Sample Size1 @romanocog @jonstrohl

#modevux

UX Test with Eye Tracking Demo

37


FLEET •  Beta •  Flyfleet.org Features: •  Real-time flight status updates •  Let friends and family track your flight •  Get delay times at check-in and

security •  Find stores and restaurants near you

Fleet was designed and developed by Rob Youmans, Daniel Gartenburg, Melissa Smith, Peter Lee, Jordan Higgins, Octavian, and Vincent Mamo

38

Example Methodology – Summative Evaluation Participants: •  N = 74 | Average Age = 37 •  Mix of gender, ethnicity, income. •  Summative Evaluation with a between

groups design. •  New, Old, Prototype, and Bilingual

media diaries.

User Experience Testing session: •  Participants read a description of the

study. •  The moderator gave instructions and

calibrated the eye tracker. •  Participants completed Steps 1-5 in the

diary at their own pace. •  End-of-session satisfaction questionnaire. •  Debriefing interview.

Eye Tracker

Moderators worked from another room.

Control Room

Slide from: Walton, L., Romano Bergstrom, J., Hawkins, D. & Pierce, C. (2014). User Experience and Eye-Tracking Study: Paper Diary Design Decisions. Paper presentation at the American Association for Public Opinion Research (AAPOR) Conference, Anaheim, CA, May 2014.

39

1.Calculations derived from: Sauro, J., & Lewis, J. R. (2012). Quantifying the user experience: Practical statistics for user research. Elsevier. Bojko, A. (2013). Eye Tracking the User Experience. Rosenfield Media.

Summative Studies Ø  Are <10 participants appropriate?

•  Very rarely, but sometimes when trying to detect very large differences. •  Difference between groups is 60% or more. •  Assuming our confidence goal is 90%. •  Assuming our power is 80%.

Ø  Are >20 participants appropriate? •  Yes, very often. Typical for detecting moderate differences with

confidence. •  Difference between groups is 30% or more. •  Assuming our confidence goal is 90%. •  Assuming our power is 80%.

Ø  If eye tracking, over recruit by ~20%

Determining Sample Size1 @romanocog @jonstrohl

#modevux

Example Methodology – Naturalistic environment

40

Participants: •  N = 8 •  Mix of gender, income. •  Participants use their own

device and Internet connectivity.

User Experience Testing session: •  Sessions occurred in participant’s home. •  The moderator gave instructions. •  Sessions began with an interview about

how participants use the Internet. •  Participants completed a set of tasks

provided individually by the moderator. •  Debriefing interview.

What can we learn from eye tracking?

42

Fixation count heat map of 10 seconds of interaction with the page. 20 participants included.*

Fixation count heat map of 10 seconds of interaction with the page. 10 Novice participants included.*

Fixation count heat map of 10 seconds of interaction with the page. 9 Experienced participants included.*

All Participants Novice Experienced

*Heatmaps include participants who located the link from the Refunds page only. Fixation count refers to the number of times the eye fixates, or directs itself towards, an area of the page.

Novice and experienced users focus on different elements.

43

Making something large does not mean people will notice it.


44

New/Old DVR Grid New/Old DVR Grid Prototype Grid

Grids that match users’ mental model are easier to process.

Walton, L., Romano Bergstrom, J., Hawkins, D., & Pierce, C. (2014). User experience testing and eye tracking inform paper diary design. Proceedings from the Human Computer Interaction International Conference, June 2014, Crete, Greece.


45

Gaze plot of 60 seconds of engagement from four participants on the Loan Rehabilitation page who scanned the text without reading it thoroughly.

Gaze plot of 60 seconds of engagement from four participants on the Contact Your Servicer page who read most of the text.

People do not read dense text.


46

The text within the orange circle indicates the information to enter on the page. The red circled area is where the user enters the ID number. The participant skips the instructions and introductory text and moves on to the actionable areas.

People do not read instructions.

He, J., Siu, C., Chaparro, B. & Strohl, J. (2014). Mobile. In J. Romano Bergstrom & A. Schall (Eds.), Eye Tracking in User Experience Design. San Francisco, CA: Morgan Kaufmann.


47

If the mandatory fields were not completed, participants received an error message that was not helpful. This gaze plot begins after the participant clicks “Ok.” The participant looks all over the screen, searching for the missing mandatory fields. The error message did not indicate which fields were missing.

People need to know what to do next

He, J., Siu, C., Chaparro, B. & Strohl, J. (2014). Mobile. In J. Romano Bergstrom & A. Schall (Eds.), Eye Tracking in User Experience Design. San Francisco, CA: Morgan Kaufmann.


Activity

48

Awesome! I want to use eye tracking. What’s next?

50

Eye-Tracking Decision Progression

Research goals and plan Form hypotheses Will eye tracking

add value? Design the study

Collect data Clean & prepare data

Visualize qualitative data

Analyze quantitative data

51

Evidence: Younger and middle-age social media users access the sites/apps largely through smartphones, whereas older users access largely through desktop computers. It is important to test across user groups and across devices.

Older Adults Middle Age Adults Younger Adults

People fixate elements of displays differently across devices. Different user groups access via different interfaces


52

Android App Homepage Tablet Web Homepage Tablet App Homepage

People fixate elements of displays differently across devices. People attend to elements of displays differently across devices

Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data Collection Tool. Paper presentation at the AAPOR Conference, Anaheim, CA, May 2014.


Thank you!

Jen Romano Bergstrom UX Researcher Facebook @romanocog [email protected]

Jon Strohl UX Researcher Fors Marsh Group @jonstrohl [email protected]

March 23, 2015 ModevUX | Arlington VA

54

Website www.forsmarshgroup.com LinkedIn hXp://www.linkedin.com/company/fors-‐marsh-‐group Blog www.forsmarshgroup.com/index.php/blog TwiXer @forsmarshgroup

Eye Tracking the UX of Mobile: What You Need to Know

Technology

Transcript of Eye Tracking the UX of Mobile: What You Need to Know