What Did I Miss?
description
Transcript of What Did I Miss?
What Did I Miss?
Kori Inkpen John C. TangRajesh Hegde
Zhengyou ZhangSasa JunuzovicChris BrooksUniv. Saskatchewan
In-Meeting Review using Multimodal Accelerated Instant Replay (AIR) Conferencing
GoalMake meetings (and play) better with telepresence technology
Meetings are important
Do you ever ….
Partial Meeting Review95% 91%
73%
30%
Verizon Conferencing Report - 2003
What did I just miss?
DVR Functionality -> Live Conference
What makes it challenging?
Previous ResearchMeeting Capture
Cutler et al., 2002; Jian, Kim & Li, 2003; Ranjan, Birnholtz, & Balakrishnan, 2008
SummariesVideo Skimming & Summarization - Smith & Kanade, 1995; He, Gupta & Grudin, 1999; Christel, 2006; Money & Agius, 2008;
Indexing Recorded MeetingsMoran et al., 1997
Time-CompressionOmoigui et al., 1999; Orr, 1971; Wildemuth et al., 2003
Real-time CatchupTucker et al., 2010
Prototype System
AIR Conferencing
AIR Conferencing System 1/2
LIVE
AIR Conferencing System 2/2
PLAYBACK
AIR Conferencing - User Study #1
Explore the benefits (and challenges) of Accelerated Instant Replay for meetings
Experimental Design18 Participants (1 female)Three conditions
Background InterviewsTask: Live Status Meeting
3-way VideoconferenceEach person gets a chance to presentEach person gets interrupted 4 times and has to catch upRecall assessment (quiz on facts + explanations)
Transcript-Only Muted Replay Full Replay
AIR – Interview Results All occasionally miss parts of meetings
Current catch-up strategies (most indicated they do not want to disrupt the meeting)
often just listen and try to deduce what they missed, possibly interrupt with a question if they are confusedwait for a recap by the presenterbrowse meeting material to figure out what was misseddiscreetly ask someone what was missed (whisper or IM)
AIR – Interview ResultsUseful to replay what was missed during a videoconference?
Yes (10), maybe (7), no (1). “Often you miss critical conversations when you step out or are interrupted during a meeting and then you try to play catch-up during the rest of the meeting. Getting to know what was covered and who said it and the body language would put me back into the meeting very quickly.”
“It would depend mostly upon the importance of the meeting, followed by the duration of how much I missed, and finally, on how discreetly I could review the video”
AIR – Recall ResultsFull Replay enabled users to recall as much as they did when they were not interrupted.
Recall TestFacts Explanations
Baseline 78% 43%
Transcript only 45% 16%
Muted Replay 40% 13%
Full Replay 80% 49%
AIR – Preference Results
Current System
Perfect Speech-to-
TextTranscript Only 2 7Muted Replay 0 2Full Replay 16 6
–
Full Replay:“it was fast, easy to concentrate and auto catch-up”“easiest to follow; leads most smoothly into rejoining live”“seems like the only way to catch up in a focused way”“fast audio was really cool …. I feel like I can cheat in time with fast forward”
AIR – Divided Attention Results
Very Difficult
Somewhat Difficult
Neither Easy nor Difficult
Somewhat Easy
Very Easy
6 8 2 2 0
Past + Present:“I can listen to audio while watching the live slide show and transcript”“you can listen to current conversations and read transcripts”
New Questions1. How important were the additional
modalities (over just audio alone)?
2. How successful would the system be with perfect speech-to-text transcription?
3. How can we better support multitasking?
AIR Conferencing - User Study #2
Examining individual modalities and impact of perfect speech-to-text
transcription
Prototype System
Videos
SharedDesktop
Speech-to-text Transcript
Experimental Design58 Participants (25 female)Pre-recorded status meeting Mixed Design
Everyone did audio only + 1 other conditionFocused review (no divided attention)
ModalitiesTranscrip
t onlyAudio only
Audio + Transcrip
tAudio + Video
Audio + Shared
Workspace
Audio + All
(full replay)
User study #2 – Recall resultsRECALL Fact Explanation Identification
Live 93% 77% 84%Audio Only 74% 50% 61%
Enhanced Audio 83%1 62%1 72%1
1 Significantly better than audio only but worse than live
User study #2 – Recall resultsRECALL Fact Explanation Identification
Live 93% 77% 84%Audio Only 74% 50% 61%
Audio + All 89%1 70%1 81%1
Audio + Workspace 90%1 65% 72%Audio + Transcript 83% 67%2 66%
Audio + Video 70% 48% 68%
1 Significantly better than audio only and no difference with live
2 Significantly better than audio only but worse than live
User study #2 – Recall resultsRECALL Fact Explanation Identification
Live 93% 77% 84%Audio Only 74% 50% 61%
Audio + All 89%1 70%1 81%1
Audio + Workspace 90%1 65% 72%Audio + Transcript 83% 67%2 66%
Audio + Video 70% 48% 68%
1 Significantly better than audio only and no difference with live
2 Significantly better than audio only but worse than live
Users were significantly more confident with their answers in
the Audio + All condition (p<.05).
User study #2 – Recall resultsRECALL Fact Explanation Identification
Live 93% 77% 84%Audio Only 74% 50% 61%
Audio + All 89%1 70%1 81%1
Audio + Workspace 90%1 65% 72%Audio + Transcript 83% 67%2 66%
Audio + Video 70% 48% 68%
1 Significantly better than audio only and no difference with live
2 Significantly better than audio only but worse than live
“I used the audio to see who was talking, and every time I missed something, I had the transcript which kept a recording of everything and I could just look back at it.”
“Being able to see who was talking during catch-up helped to associate a face, name, and voice with the answers given.”
User study #2 – Transcript OnlyRECALL Fact Explanation IdentificationAudio Only 71% 50% 56%
Transcript Only 77%1 52% 43%2
1 Significantly better than audio only
2Significantly worse than audio only
User study #2 – Transcript OnlyRECALL Fact Explanation IdentificationAudio Only 71% 50% 56%
Transcript Only 77%1 52% 43%2
1 Significantly better than audio only
2Significantly worse than audio only
Transcript alone was way too fast to really understand what was going on. I could kind of skim, and I got some information but I felt like I was just bouncing along.
AIR Conferencing Conclusions
Enhanced audio catch-up is superior to audio-only review.
Users preferred, felt more confident with, and performed better with enhanced-audio than audio-only review. As good as live!Results consistent between two studies
Audio + All showed the strongest benefitAudio + Shared Workspace improved recall of factsAudio + Transcript improved recall of explanations
Speech-to-Text transcript not as beneficial as expected
AIR Conferencing ConclusionsCost/benefit tradeoff:
importance of information vs. overhead Ensure seamless use so as to not detract from the meeting
Next Steps:Enhanced timeline indexMulti-taskingAIR for face-to-face meetings
(with mobile phones/laptops & headsets)
AIR VisionReplaying life should be “a given”
But sometimes participation in real-time is important, so we need to enable catch-up
Thank you!
Questions?