Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Analyst)
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures
-
Upload
jennifer-romano-bergstrom -
Category
Technology
-
view
607 -
download
0
Transcript of Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures
![Page 1: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/1.jpg)
Effects of Age and
Think-Aloud Protocol on
Eye-Tracking Data and
Usability Measures
Jennifer Romano Bergstrom @romanocog
Fors Marsh Group @forsmarshgroup
&
Erica Olmsted-Hawala
U.S. Census Bureau
EyeTrackUX | Las Vegas, NV | June 2012
![Page 2: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/2.jpg)
Usability Testing
• Technique in user-centered design to
evaluate a product by testing it with actual
users of the product
• Value is getting direct feedback on how
real users work with the product
![Page 3: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/3.jpg)
How Usability is Measured
• International Organization for
Standardization (ISO) 9242:11
The extent to which a product can be used by
specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a
specified context of use.
![Page 4: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/4.jpg)
Usability Testing
• Metrics
– Task accuracy
– Task efficiency
– User satisfaction
• Usability professional (test administrator)
identifies problem areas of product based
on user performance & verbalized
feedback
![Page 5: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/5.jpg)
5
Think-Aloud Protocol
• Primary tool used by usability professionals when conducting usability tests
• Test administrator — Listens to what participant says — Watches what participant does — Cannot directly observe what a participant
thinks
![Page 6: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/6.jpg)
6
Concurrent Think Aloud (CTA)
Protocol
• Participant encouraged to “think aloud” while working on task – Running commentary of thought-action
process – Participant articulates as they work on task
![Page 7: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/7.jpg)
Retrospective Think-Aloud (RTA)
Protocol
• Participant completes session in silence
– Watches a video replay of session
• Sometime with eye tracking
– Articulates while watching video replay
![Page 8: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/8.jpg)
Research on CTA and RTA
• Usability studies have compared effects of
CTA and RTA on usability performance
– Accuracy
– Efficiency
– Satisfaction
– Verbal data quality
• Results vary by study
![Page 9: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/9.jpg)
Pros and Cons of Think Aloud CTA
• Metrics might not be accurate
• Emotions
• Users describing the what
• Real sense of users’
experience with site
• Shorter sessions
• Real time responses
RTA
• Metrics “more pure”
• Reflection
• Users describe their own
interpretation or
understanding of why
• Users’ insight into what the
design issues are
• Longer sessions
• Memory is fallible
![Page 10: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/10.jpg)
Cognitive Aging
![Page 11: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/11.jpg)
Aging: The Facts
• The population is increasing.
![Page 12: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/12.jpg)
The Facts
250,000
270,000
290,000
310,000
330,000
350,000
370,000
390,000
2000 2005 2009 2010 2015 2020 2025 2030
Po
pu
lati
on
of
US
in t
ho
usa
nd
s
![Page 13: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/13.jpg)
The Facts
• The population is increasing.
• Older adults are the fastest growing part of
the population.
![Page 14: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/14.jpg)
The Facts
0
20,000
40,000
60,000
80,000
100,000
120,000
2000 2005 2009 2010 2015 2020 2025 2030
Po
pu
lati
on
of
US
in t
ho
usa
nd
s
18 to 24 years
25 to 44 years
45 to 64 years
65 years and over
![Page 15: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/15.jpg)
The Facts
Percent Change in US Population and in Older Adults: 2000-2029.
![Page 16: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/16.jpg)
The Facts
• The population is increasing.
• Older adults are the fastest growing part of
the population.
• More people are using the Internet.
![Page 17: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/17.jpg)
The Facts
0
10
20
30
40
50
60
70
80
90
1997 2000 2001 2003 2007 2009
Pe
rce
nt
of
ho
use
ho
lds
wit
h In
tern
et
![Page 18: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/18.jpg)
The Facts
![Page 19: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/19.jpg)
The Facts
• The population is increasing.
• Older adults are the fastest growing part of
the population.
• More people are using the Internet.
• Cognitive decline is inherent with age.
![Page 20: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/20.jpg)
The Facts
• The population is increasing.
• Older adults are the fastest growing part of
the population.
• More people are using the Internet.
• Cognitive decline is inherent with age.
![Page 21: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/21.jpg)
Cognition and Internet Examples
• Spatial skills
– Where things are located
• Short-term memory
– Remembering the purpose
• Working memory
– Remembering information entered earlier to respond to later questions
• Sustained attention
– Complex Web sites
![Page 22: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/22.jpg)
The Facts
• The population is increasing.
• Older adults are the fastest growing part of
the population.
• More people are using the Internet.
• Cognitive decline is inherent with age.
• Web site designers often do not take
cognitive decline into account when
designing Web sites and surveys.
![Page 23: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/23.jpg)
Cognitive Aging,
Think-Aloud Protocol
and Usability Performance
![Page 24: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/24.jpg)
Challenges of Dual tasks
• Performance for older adults decreases
when they attempt to complete dual tasks
![Page 25: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/25.jpg)
Research Questions
• Does the dual task of thinking aloud while
working on finding information impact
usability performance and eye fixation
patterns?
• What think-aloud protocol (CTA / RTA)
should practitioners use when conducting
usability studies with older adults?
![Page 26: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/26.jpg)
Our Study
• 2 (CTA / RTA) x 3 (Age) between-subjects
study.
• Outcome Variables
– Accuracy
– Efficiency
– Satisfaction
– Eye-tracking data
– Verbal reports
![Page 27: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/27.jpg)
Participants
• 95 participants
• Metro DC area
• Three age groups
– Young adults ages 18-28
– Middle-age adults ages 40-50
– Older adults ages 64-76
• One year of prior experience using Internet • Reimbursed $40.00
![Page 28: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/28.jpg)
Participant Characteristics
Young adults Middle-age adults Older adults
CTA RTA CTA RTA CTA RTA
Gender 3M / 8F 6M / 15F 5M / 7F 9M / 10F 5M / 6F 9M / 12F
Mean Age 22 22 45 46 67 68
Mean Years of
education 15 15 15 14 16 16
Ease of learning
a new Web site* 1.36 1.47 1.83 1.42 2.36 2.38
Ease of
navigating the
Internet*
1.09 1.05 1.33 1.16 1.72 1.57
*Scale: 1 (Not difficult at all) – 5 (Extremely difficult).
![Page 29: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/29.jpg)
Methods • 5 information-gathering tasks
– Typical tasks users come to site to accomplish
– Tasks organized by easy to hard
– Collected accuracy and efficiency measures
– Eye tracking on all tasks
• Satisfaction questionnaire and debriefing at end
• CTA group – CTA on all tasks
• RTA group – RTA on last task only
– Gaze video playback
• Test administrator and participant were in separate rooms
![Page 30: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/30.jpg)
Website Tested
• US Census Bureau’s American FactFinder
– Primary website about population, housing,
and economy of the US
– Legacy version of site
![Page 31: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/31.jpg)
AFF Website: http://factfinder.census.gov
![Page 32: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/32.jpg)
Example Tasks
• First easy task: You want to learn more
about Maryland, and specifically about
how many people live there. How many
people live in Maryland?
• First hard task: You are doing a report on
schooling in the U.S. What percent of the
population in Florida, California and Texas
completed college in 2008?
![Page 33: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/33.jpg)
Results
• Task accuracy
• Task efficiency
• User satisfaction
• Eye Tracking
• Verbal reports
![Page 34: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/34.jpg)
Young and middle age adults
completed more tasks successfully
than older adults
0%
10%
20%
30%
40%
50%
60%
70%
80%
Young adults Middle-age adults Older adults
CTA
RTA
**
*
*p < 0.05; **p < 0.01
![Page 35: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/35.jpg)
Middle-age adults in CTA had higher
accuracy in hard task than in RTA
0%
10%
20%
30%
40%
50%
60%
70%
80%
Middle-age adult--1st hard task
CTA
RTA
*
*p = 0.07
![Page 36: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/36.jpg)
Young adults completed tasks
faster than older adults
0:00
0:28
0:57
1:26
1:55
2:24
2:52
3:21
3:50
4:19
Young adults Middle-age adults Older adults
CTA
RTA
**
**p = 0.05, *p = 0.10
![Page 37: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/37.jpg)
Middle-age adults in RTA completed
hard task faster than in CTA
0:00
1:12
2:24
3:36
4:48
6:00
7:12
8:24
9:36
10:48
Middle age adults--1st hard task
CTA
RTA
*
*p = 0.10
![Page 38: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/38.jpg)
No significant age-related
differences in user satisfaction
0
1
2
3
4
5
6
7
8
9
Young adults Middle-age adults Older adults
CTA
RTA
![Page 39: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/39.jpg)
Speed / Accuracy Trade Off
• Cognitive demands influence the speed
and accuracy with which adults process
information
• Our findings for the middle-age adults are
consistent with earlier cognitive work.
– Higher accuracy for hard task with CTA
– Slower for hard task with CTA
![Page 40: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/40.jpg)
Interaction of Age and Think Aloud
• Older adults’ performance decreases
when they perform dual tasks.
• Our findings for the middle age adults are
consistent with earlier cognitive work.
– Older adults: floor level
– Young adults: ceiling level
![Page 41: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/41.jpg)
Eye Tracking
• Total number of fixations per AOIs
– Task 1
• Total number of fixations on main page
– Task 1
![Page 42: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/42.jpg)
AOIs
Left navigation Population clock
Top banner Top navigation
Whole screen
![Page 43: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/43.jpg)
Left navigation: No difference
between CTA and RTA
0
5
10
15
20
25
CTA RTA
![Page 44: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/44.jpg)
Left navigation: Older adults had
fewer fixations than young adults
for CTA only *
*p < 0.05
0
1
2
3
4
5
6
7
8
9
10
Young adults Middle-age adults Older adults
CTA
RTA
![Page 45: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/45.jpg)
Top banner: More fixations during
CTA
0
1
2
3
4
CTA RTA
CTA
RTA
p = 0.06
![Page 46: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/46.jpg)
Top banner: No significant
differences, but pattern is the same
0
1
2
3
4
5
Young adults Middle-age adults Older adults
CTA
RTA
![Page 47: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/47.jpg)
Top navigation: No difference
between CTA and RTA
0
1
2
CTA RTA
CTA
RTA
![Page 48: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/48.jpg)
Top navigation: Older adults had
fewer fixations than young adults
for CTA only *
*p < 0.05
0
1
2
3
4
Young adults Middle-age adults Older adults
CTA
RTA
![Page 49: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/49.jpg)
Population clock: No difference
between CTA and RTA
0
1
2
3
4
5
6
CTA RTA
CTA
RTA
![Page 50: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/50.jpg)
Population clock: No significant
differences, but CTA pattern is similar
0
1
2
3
4
5
6
7
8
Young adults Middle-age adults Older adults
CTA
RTA
![Page 51: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/51.jpg)
Whole screen: No difference
between CTA and RTA
0
10
20
30
40
50
60
CTA RTA
CTA
RTA
![Page 52: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/52.jpg)
Whole screen: Middle-age adults
had fewer fixations than young
adults for CTA only
*
*p = 0.05
0
10
20
30
40
50
60
70
80
Young adults Middle-age adults Older adults
CTA
RTA
![Page 53: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/53.jpg)
Seconds to complete Task 1: No
significant differences
0
20
40
60
80
100
120
140
160
Young adults Middle-age adults Older adults
CTA
RTA
![Page 54: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/54.jpg)
Accuracy for Task 1: No significant
differences
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Young adults Middle-age adults Older adults
CTA
RTA
![Page 55: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/55.jpg)
Language Analysis
• Present tense and past tense
verbalizations
• Also examined
– Positive emotion words (e.g., good, helpful)
– Words of assent (e.g., absolutely, cool)
– Negative emotion words (e.g., annoy, awful)
– Negations (e.g., aren’t, can’t)
– Affective processing (e.g., caring, insult)
![Page 56: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/56.jpg)
Participants in CTA spoke more present tense;
Participants in RTA spoke more past tense
0%
2%
4%
6%
8%
10%
12%
14%
Past tense Present tense
CTA
RTA
![Page 57: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/57.jpg)
Young adults in CTA spoke 5%
fewer utterances in present tense
0%
2%
4%
6%
8%
10%
12%
14%
16%
Present tense
Young CTA
Middle-age CTA
Older CTA
![Page 58: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/58.jpg)
Young adults in RTA follow similar pattern:
fewer utterances of present tense
0%
2%
4%
6%
8%
10%
12%
Present tense
Young RTA
Middle-age RTA
Older RTA
![Page 59: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/59.jpg)
Words of Insight and Cognition
• Test administrator observations during sessions – CTA more immediate what they were doing
– RTA reflections on actions
• Insight words – Complex
– Meaning
– Prove
• Cognitive processes – Ambiguous
– Hesitate
– Solution
![Page 60: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/60.jpg)
Participants in RTA spoke more
words of insight and cognition
0%
5%
10%
15%
20%
25%
Insight words Cognitive processes
CTA
RTA
![Page 61: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/61.jpg)
Summary of Usability Results I
• Accuracy – Young and middle-age adults
completed more tasks successfully (higher
accuracy) than older adults.
• Efficiency – Young adults completed tasks
faster than older adults.
• Satisfaction – Age did not affect
satisfaction.
![Page 62: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/62.jpg)
Summary of Usability Results II
• Condition effect – Middle-age adults who
completed tasks in silence (RTA)
completed the most difficult task faster and
with higher accuracy than middle-age
adults who thought aloud (CTA) while
completing tasks.
![Page 63: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/63.jpg)
Summary of Eye-Tracking Results
• When age was not a factor, there appeared
to be no differences in fixations between CTA and RTA
• When age was taken into account – Older had fewer fixations than young in CTA only
• Left navigation
• Top navigation
• Similar pattern for other AOIs
– Middle-age had fewer fixations than young in CTA only
• Whole screen
– Not related to accuracy or efficiency
![Page 64: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/64.jpg)
Implications
• Recruit participants of varying age groups for usability
testing; conduct usability tests with older adults.
• Be aware of cognitive demands and effects that thinking
aloud has on aging adults.
• Consider pros and cons when creating a protocol and
selecting a think-aloud method.
– Use CTA for a richer verbal commentary, particularly with respect to
participants’ emotional reaction to the site.
– Use RTA when interested in verbal feedback related to participants’
insight on the issues they had with the application.
– Use RTA when pure eye-tracking data is necessary.
![Page 65: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/65.jpg)
Future Analysis
• Fixation duration
• Sum across all tasks – Easy vs. hard tasks
• Performance in first 10 seconds
• Correlations – Education
– Experience
– Cognitive abilities
• Verbalizations – Phrases
– Number of words, phrases
• Click data
• Identified usability issues
![Page 66: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/66.jpg)
Future Research
• Include wider ranges of age to test the
assertion that the age-related effect is
incremental and occurs slowly through a
lifetime
• Alter RTA protocol
• Replicate with different web sites
• Are there specific gaze patterns by age
and think-aloud protocol?
![Page 67: Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Measures](https://reader033.fdocuments.net/reader033/viewer/2022052410/554f07d8b4c90577698b52dc/html5/thumbnails/67.jpg)
Thank You!
Jennifer Romano Bergstrom
Fors Marsh Group
571.858.3795
@romanocog
Proceedings paper from STC re verbalizations is available.
Eye-tracking paper is forthcoming.