Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute
description
Transcript of Carolyn Penstein Rosé Language Technologies Institute Human-Computer Interaction Institute
1
Carolyn Penstein RoséLanguage Technologies InstituteHuman-Computer Interaction InstituteSchool of Computer Science
With funding from the National Science Foundation and the Office of Naval Research
Souflé: A Three Dimensional Framework for Analysis of Social Positioning
in Dyadic and Group Discussions
2
Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…
Human learning
Health
Wellbeing
3
Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…
Human learning
Health
Wellbeing
Human learning
4
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
5
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
Purpose of Interaction in Learning Contexts
Reward structures encourage students to focus on performance over learning
Well crafted instruction provides opportunities for learning
Opportunities only help if students take them
Take Home Message:
Introducing reflection points provides opportunities for students to take advantage of learning resources
CarefullyStructuredConceptualKnowledge
Important Ingredients for Learning
7
Reflectionthrough
RichInteraction
End of Fall Semester: Students learn about Rankine Cycles 1 Week of lectures Homework assignment on analysis of
Rankine Cycles Tutorial on using CyclePad software
package (Developed at Northwestern University (Forbes et. al. 1999) Allows students to construct and
analyze a variety of Thermodynamic Cycles)
Instructed on Effects of Changing System Variables (Temperature, Pressure) on System Output (Power, Waste Heat)
Second-Year Thermodynamics
Learning Goal: Encourage students to reflect on interactions between cycle parameters
Collaborative Task
Reduction in Steam Quality
Power
Waste Heat
Increasing heat increases power but also waste heat
Increasing pressure increases efficiency
Design Goal: Design a power plant based on the Rankine Cycle paradigm
Competing Student Goals: Power: Design a power plant that achieves
maximum power output Motivated by economic concerns
Green: Design a power plant that has the minimum impact on the environment Motivated by environmental concerns
Each pair turns in exactly one design
Computer Supported Collaborative Learning
11
12
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
Souflé Framework(Howley et al., in press)
13
Person Person
3 Dimensions: Transactivity Engagement Authoritativeness
Souflé Framework(Howley et al., in press)
14
Sociolinguistics
Discourse Analysis
LanguageAnd
Identity
LanguageUse
MachineLearning
Multi-Level
Modeling
AppliedStatistics
ComputationalModels
OfDiscourseAnalysis
Souflé Framework(Howley et al., in press)
15
Transactive Knowledge Integration
Person Person
16
i
• Definition of Transactivity• building on an idea expressed earlier in a conversation • using a reasoning statement
We don't want tmax to be at 570 both for the material and [the Environment]
well, for power and efficiency, we want a high tmax, but environmentally, we want a lower one.
17
18
19
Findings Moderating effect on learning (Joshi & Rosé, 2007; Russell, 2005; Kruger &
Tomasello, 1986; Teasley, 1995)
Moderating effect on knowledge sharing in working groups (Gweon et al., 2011)
Computational Work Can be automatically detected in:
Threaded group discussions (Kappa .69) (Rosé et al., 2008)
Transcribed classroom discussions (Kappa .69) (Ai et al., 2010)
Speech from dyadic discussions (R = .37) (Gweon et al., 2012)
Predictable from a measure of speech style accommodation computed by an unsupervised Dynamic Bayesian Network (Jain et al., 2012)
Transactivity (Berkowitz & Gibbs, 1983)
Engagement Engagement
Souflé Framework(Howley et al., in press)
21
Transactive Knowledge Integration
Person Person
Engagement(Martin & White, 2005, p117)
System of Engagement Showing openness to the
existence of other perspectives
Less final / Invites more discussion
Example: [M] Nuclear is a good choice [HE] I consider nuclear to be
a good choice [HC] There’s no denying that
nuclear is a superior choice [NA] Is nuclear a good
choice?22
23
Findings Correlational analysis: Strong correlation between displayed openness of
group members and articulation of reasoning (R = .72) (Dyke et al., in press)
Intervention study: Causal effect on propensity to articulate ideas in group chats (effect size .6 standard deviations) (Kumar et al., 2011)
Mediating effect of idea contribution on learning in scientific inquiry (Wang et al., 2011)
Engagement (Martin & White, 2005)
Authority
Authority
Engagement Engagement
Souflé Framework(Howley et al., in press)
25
Transactive Knowledge Integration
Person Person
Analysis of Authoritativess
26
Water pipe analogy: Water = Knowledge or Action Source = Authoritative speaker Sink = Non-authoritative Speaker
Authoritativeness Ratio = Source Actions Actions
27
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
28
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
29
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
The Negotiation Framework(Martin & Rose, 2003)
31
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
32
The Negotiation Framework(Martin & Rose, 2003)
Source
or Sink?
Primary
Se
condary Type of Content?
Knowledge Action
K2requesting knowledge,
information, opinions, or facts
K1giving knowledge, information,
opinions, or facts
A2Instructing, suggesting, or
requesting non-verbal action
A1Narrating or performing your
own non-verbal action
Additionally…ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
K1 + A2
K1 + K2 + A1 + A2
Authoritativeness:
33
K2?
34
Set up!
K1
K2
35
Findings Authoritativeness measures display how students respond to
aggressive behavior in groups (Howley et al., in press) Authoritativeness predicts learning (R = .64) and self-efficacy (R = .35)
(Howley et al., 2011) Authoritativeness predicts trust in doctor-patient interactions (R
values between .25 and .35) (Mayfield et al., under review)Computational Work
Detectable in collaborative learning chat logs (R = .86)
Detectable in transcribed dyadic discussions in a knowledge sharing task (R = .95) (Mayfield & Rosé, 2011)
Detectable in transcribed doctor-patient interactions (R = .96) (Mayfield et al., under review)
Authoritativeness (Martin & Rose, 2003)
37
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
Computer Supported Collaborative Learning
39
AutomaticAnalysis
Of Conversation
ConversationalInterventions
PositiveLearning
Outcomes
6 Years of Positive Results
Foundational study: students work with a partner and dialogue agent for support Learn 1.24 s.d. more than individuals without support (Kumar et al., 2007a)
Results inform iterative design of agent behavior Personalized agents increase supportiveness and help exchange between students (Kumar et al.,
2007b) Agents are more effective when students have control over timing of the interaction (Chaudhuri
et al., 2008; Chaudhuri et al., 2009) Agents that employ Balesian social strategies are more effective than those that do not (Kumar
et al., 2010; Ai et al., 2010) Students are sensitive to agent rhetorical strategies such as displayed bias (Ai et al., 2010),
displayed openness to alternative perspectives (Kumar et al., 2011), and targeted elicitation (Howley et al., 2012)
Bazaar architecture enables efficient, principle based agent development (Kumar & Rosé, 2011; Adamson & Rosé, 2012)
41
• Introducing the Problem of Supporting Productive Discussion for Learning
• Discussion of Souflé• Transactivity• Engagement• Authoritativeness
• Application to Dynamic Support for Group Learning
• Conclusion and Current Directions
Outline
42
• Transactivity is a conversational behavior that is important for learning
• Authoritativeness and Engagement are dimensions of conversation that play a supporting role
• Positioning students to exchange Transactive contributions
• We have made progress at automating analysis of Transactivity and Authoritativeness
•Automated analysis enables dynamic triggering of supportive interventions for online group learning
Conclusions and Current Directions
43
•In the future, CSCL activities will be part of societies of online learners
• Vision began with Virtual Math Teams/ The Math Forum• Now we’re already seeing the shift through companies
like Coursera and Udacity
•We are taking steps towards this future• Fully distance learning studies (UCSB, Drexel)• Sustainable CSCL (online office hours agent)
•As we look to the future: we must understand the emergent effects of our local interventions in order to maximize positive benefit on a grand scale
Conclusions and Current Directions
44
Thank You!
Questions?
Revoicing Agent
45
46
Experimental Paradigm Day 1: Pretest on conceptual questions related to the unit (Diffusion or
Punnett Squares) Day 2: Online collaborative activity + Immediate Posttest isomorphic to
pretest Day 3: Whole class teacher led discussion + Delayed Posttest isomorphic to
pretestParticipants: consenting 9th grade biology students, randomly
assigned to groups of 3Experimental Design: Simple between subjects design
Groups randomly assigned to Revoicing condition or Control Condition
Revoicing Agent Studies
Revoicing Agent
47
48
Study 1: Year 1, Diffusion Lab (50 students) Students learn more on explanation questions in supported conditions
(F(1,46) = 4.3, p < .05, effect size 1 standard deviation) Students in supported conditions more active in whole group discussion
(F(2,26) = 4.2, p < .05, effect size .75 standard deviations)
Study 2: Year 2, Diffusion Lab (78 students) Students learn more on immediate post test in Revoicing Agent
condition (F(1,74) = 4.3, p < .05, effect size .51 standard deviations)
Study 3: Year 2, Punnett Square Lab (78 students) Students learned more on delayed post-test in Revoicing Agent
condition
Results of CSCL Studies