TER Workshop J P San Diego
-
Upload
jonathan-san-diego -
Category
Education
-
view
763 -
download
0
description
Transcript of TER Workshop J P San Diego
Examining Learner-computer Interactions: Advanced Lab-based Research Methods
Slides before 1st Section
Divider
Motivation of the research
Strategy as a unit of
analysis
Illustrative Analyses and
Some Findings
Unused Section Space 2
Technology Enhanced Research
Unused Section Space 1
Advanced lab-based methods
Illustrative study
J.P. San Diego and J.C. Aczel
Outline• Examining learner-computer interactions
– Focus on detecting learning, even when nothing is being explicitly “taught”
– Trying to understand why and how learning is occurring– Within the learning context– Largely visual
• Advanced lab-based research methods– Not just pointing a camera at a screen– Or asking “what are you learning?”– Advanced data collection: eye-tracking, sketches, gestures,
physiological measures– Advanced data analysis: handling multiple video streams,
software analytics and strategy as unit of analysis
Outline• Based on a specific study
– PhD research– Will broaden this out to reflections of other methods
and applications
Acknowledgements• Dr. James Aczel, Dr. Barbara Hodgson and Prof.
Eileen Scanlon• Prof. Josie Taylor and Dr. Richard Cox• Prof. Marian Petre, Prof. John Mason, Dr. Ann
Jones, Dr. Patrick McAndrew and Dr. Denise Whitelock
• Dr. Ekaterini Tzanidou, Dr. Geke vanDijk, Dr. Miquel Prats and Ms. Anesa Hosein
• The participants of the study
AcknowledgementsProf. Diana Laurillard and Prof. Margaret CoxPascal Mangold of Mangold Software & Consulting
GbHM Microsoft Research in Cambridge through Dr. Fabien
PetitcolasIET-IT (Will Woods, P. Downs, D. Perry & S.
Hammond) and CALRG ColleaguesOU-LTS (Mr. Collin Thomas) Prof. Jeff Johnson, Prof. Chris Earl, Dr. Peter Lloyd
and Dr. Georgy Holden,
The 2003 MSc Study
Methodological challenges
“You need times ‘cause you need it to that (points on the screen) times twenty”
“Oh OK I can see what it is doing (the graph) It is going towards there”
Traditional approaches to analysing video data
• Methodological– Reflexivity (e.g. Camera effect), selectivity (transcript
as data versus video as data)
• Technical– Selecting, setting up, and operating video equipment
• Practical– Data storage, transcription and coding
• Ethical– Anonymity and privacy
Digital Video and digital data
• Advantages– Consistent record than observation notes, capture
difficult-to-record events, multi-perspective, multi-observers, offers flexibility, stimulus for discussion
• Recent developments– Variety of media, logs in video forms, video search
technologies, processing power of computers, sensors, eye-tracking, haptics, sketch recognition, etc.
The data capture setup
19 February 2007 LKL Seminar ([email protected]) 12
Data capture and analysis tools
INTERACT™
Protocols• Think-aloud
– Ericsson and Simon (1984)• Eye-tracking
– Yoon and Narayanan (2003), Hansen et al . (2001)• Sketching
– Pirrie (1996, 1997), Cox (1996)
The study design
• Data collection– 18 students with A-level Maths or higher
• 3 comparable tasks– External mathematical representations
• Each task presented in either static, dynamic and interactive forms
Standard external representations and instantiations
Instantiations
• Static: Non-moving, non-changing, non-interactive
• Dynamic: Capable of animation through alpha-numeric inputs
• Interactive: Directly manipulable graphs
The data
JSD.mov
Main research question• How do representations instantiated in
different ways influence learners’ cognitive processes?
Strategies• strategic theories
– strategies are attempts to solve problems– theories are conjectured expectations, dispositions, or assumptions
(articulated or not), of some sort of reality in a particular context– a strategy can be considered as theoretical, in a sense, in that it incorporates
expectations about some state of affairs– theory can be considered strategic, in a sense, in that some are
instrumentally better adapted to reality than others• Donald T. Campbell: Blind-Variation-and-Selective-Retention
– a mechanism for introducing variation [thought trials];– a consistent selection pressure [concerns]– a mechanism for preserving and reproducing the selected variations
• Learning– processes of discontinuous trial-and-improvement of strategic theories under
the selection pressures provided by concerns
Hypotheses• Strategies with each standard external representation can be
characterised at different levels of granularity.• Learners’ choice of strategies depends not just on the standard
external representations given but also on the instantiation.• Mental constructions of images with graphical representations
vary between instantiations.• Attention paid with each standard external representation
varies between instantiations. • Expression of inferences varies depending on the instantiation• Analyses of strategies based on gazes, actions, utterances and
sketches can identify factors contributing to strategy choice in a way that is not possible with traditional observation techniques.
Strategies identified
• Representation-specific– Algebraic, graphic and numeric
• Imagining– Pen, mouse, mental, gaze, gesture
• Re-representing– Visual, textual, symbolic
Representations-specific strategies by task X instantiation
Frequency of participants for each strategy
The chart shows the participants’ imagining strategies graphed by instantiation across the
three tasks
Areas Of Interest
Root task - Area of Interest Analysis (AOI)
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
COUNT 57.4% 10.9% 31.6% 49.6% 23.4% 27.1% 57.2% 11.3% 31.5%
DURATION 60.6% 11.4% 28.0% 45.8% 29.1% 25.1% 65.9% 9.7% 24.4%
Graphic Equations Numbers Graphic Equations Numbers Graphic Equations Numbers
Static Dynamic Interactive
“Aha! moments”
Participant’s talk: Aha! They are the same distance away.
“Invisible path”
Participant’s talk: This is going from minus two…
“Invisible region”
Participant’s talk: I’m trying to imagine what happens as the line tends to infinity…
Re-representation
Participant's talk: I don’t know what to call it… Err… I’ll just draw it
‘Freeze frames’
Attention paid to representations
Focus of attention
Findings relating to difficulties
00:14:13:22P4: It will never ever comes cross… Something... it never comes across
Bringing the evidence together
19 February 2007 LKL Seminar ([email protected]) 39
Other examples of evidence
Other examples of evidence
19 February 2007 LKL Seminar ([email protected]) 41
Participant's talk: I don’t know what to call it… Err… I’ll just draw it
• Current project– hapTEL (Haptic Technology Enhanced Learning)– PhD Student (Arash Shahriari-rad)
• TER and Formative feedback
The Future
19 February 2007 LKL Seminar ([email protected]) 43
The Future…
• Current focus on attention– Mobiles– Windows– Books
• Jo Iacovides & games– jaw tension (EMG)– skin conductance (GSR)– heart-rate (EKG)– brainwaves (EEG)
From Marvin Minsky (The Society Of Mind)
It often does more harm than good to force definitions on things we don't understand. Besides, only in logic and mathematics do definitions ever capture concepts perfectly.
The things we deal with in practical life are usually too complicated to be represented by neat, compact expressions.
Especially when it comes to understanding minds, we still know so little that we can't be sure our ideas about psychology are even aimed in the right directions.
In any case, one must not mistake defining things for knowing what they are.