Gaze-Augmented Think-Aloud as an Aid to Learning Sarah A. Vitak Scripps College Andrew T. Duchowski, Steve Ellis, Anand K. Gramopadhye Clemson University CHI May, Austin, TX John E. Ingram Sewanee: University of the South Motivation: training Visual search: well-defined strategies have been developed by experts e.g., top-down cognitive strategies based on experience Chest X-Ray (CXR) inspection: Airway, Bones, Cardiac silhouette, Diaphrams, External tissue,... Expert (left) and novice (right) scanpaths over abnormal CXR. Objectives: histology Develop histological search strategy training Test effectiveness of experts gaze atop video scanpath needs to demonstrate experts search strategy Task: identify BrdU marked cells in epithelial layer Example stimuli: immuno-gold (BrdU) stained cross-sections of bovine mammary tissue. Previous work: tracking experts Ericsson et al. (2006) surveyed experts gaze: 1. experts search strategies are task dependent 2. experts shorter dwell times thought to reflect expertise 3. experts make better use of peripheral information 4. experts patterns of visual analysis develop with training 5. experts use a larger area around fixation 6. experts make better use of extra-foveal information Expert football player (Ronaldo, gaze captured with Dikablis tracker), expert pilots (Weibel et al., 2012 [ETRA]), expert laparoscopic surgeons (courtesy Stella Atkins and Bin Zheng) Previous work: scanpath training Fertile research area (see paper for review) Selected references related to training: static, stylized scanpaths (VR) Sadasivan et al. (2005) dynamic scanpaths (still images) Nalangula et al. (2006) static scanpaths (still images) Litchfield et al. (2010) dynamic scanpaths (video) Jarodzka et al. (2010) Sadasivan et al. (2005). Jarodzka (2010). Contributions: GATA Gaze-Augmented Think-Aloud (GATA) builds on: Feed-forward training Sadasivan et al. (2005) hierarchical task analysis & scanpath visualization Stimulated Retrospective Think-Aloud Guan et al. (2006) verbalizing while watching scanpath visualization Eye Movement Modeling Examples (EMME) Jarodzka et al. (2010, 2009, 2010) task analysis & highlighting of salient regions (e.g., foveation) Gaze-Augmented Think-Aloud GATA: recorded scanpath & verbalization (audio track) a specific visual search strategy is required for histology images, devised STAMP: Staining, Tissue, Artifacts, Magnification, Plane of section Training video used in study. Evaluation Experiment: 2 x 2 mixed factorial design: (presence or absence of video) x (experienced or naive participant) video between-subjects, 8 stimuli images within-subjects task: identify cells marked with BrdU in epithelial layer procedure: both groups saw training slides w/out scanpath Training slides: the first highlights marked cells, second differentiates tissue type, third shows epithelia with no marked cells, fourth again shows marked cells in the epithelia, fifth shows no marked cells; same verbalization was heard as in GATA video. Evaluation Experiment: apparatus: Tobii ET-1750 (see paper for specs) participants: 32 (aged 19-57, median 22) 15 experienced, 15 naive 2 participants dropped (no data) dependent measures speed (time to task completion) accuracy (hits & misses) fixation counts fixation durations outliers: beyond 3 SD one removed (from A&VS) Example of participant & apparatus; half the participants (experienced) recruited from Animal & Veterinary Science (A&VS) class who were familiar with tissue images. Dependent measures Speed time to task completion tabulated per each image, then averaged Accuracy hits (true positives) misses (false positives) Fixation counts Fixation durations longer durations suggest cognitive load (e.g., difficulty) Example of two captured scanpaths: blue is from experimental group, red is from control group. Results: speed Time to completion: two-way ANOVA revealed significant effect of video F(1,6)=9.25, p