1 David Chen & Raymond Mooney Department of Computer Sciences University of Texas at Austin Learning...
-
Upload
briana-hood -
Category
Documents
-
view
213 -
download
0
Transcript of 1 David Chen & Raymond Mooney Department of Computer Sciences University of Texas at Austin Learning...
1
David Chen & Raymond MooneyDepartment of Computer Sciences
University of Texas at Austin
Learning to Sportscast: A Test of Grounded Language Acquisition
2
Motivation
• Constructing annotated corpora for language learning is difficult
• Children acquire language through exposure to linguistic input in the context of a rich, relevant, perceptual environment
3
Goals
• Learn to ground the semantics of language
Block
• Learn language through correlated linguistic and visual inputs
4
Challenge
5
Challenge
6
Challenge
A linguistic input may correspond to many possible events
Block ?
??
7
Overview
• Sportscasting task
• Tactical generation
• Strategic generation
• Human evaluation
8
Learning to Sportscast
• Robocup Simulation League games
• No speech recognition– Record commentaries in text form
• No computer vision– Ruled-based system to automatically extract
game events in symbolic form
• Concentrate on linguistic issues
9
Robocup Simulation League
10
Robocup Simulation League
Pink4’s pass was intercepted by Purple6
11
Learning to Sportscast
• Learn to sportscast by observing sample human sportscasts
• Build a function that maps between natural language (NL) and meaning representation (MR)– NL: Textual commentaries about the game– MR: Predicate logic formulas that represent
events in the game
12
Mapping between NL/MR
NL: “Purple3 passes the ball to Purple5”
MR: Pass ( Purple3, Purple5 )
Semantic Parsing (NL MR)
Tactical Generation (MR NL)
13
Robocup Sportscaster TraceNatural Language Commentary Meaning Representation
Purple goalie turns the ball over to Pink8
badPass ( Purple1, Pink8 )
Pink11 looks around for a teammate
Pink8 passes the ball to Pink11
Purple team is very sloppy today
Pink11 makes a long pass to Pink8
Pink8 passes back to Pink11
turnover ( Purple1, Pink8 )
pass ( Pink11, Pink8 )
pass ( Pink8, Pink11 )
ballstopped
pass ( Pink8, Pink11 )
kick ( Pink11 )
kick ( Pink8)
kick ( Pink11 )
kick ( Pink11 )
kick ( Pink8 )
14
Robocup Sportscaster TraceNatural Language Commentary Meaning Representation
Purple goalie turns the ball over to Pink8
badPass ( Purple1, Pink8 )
Pink11 looks around for a teammate
Pink8 passes the ball to Pink11
Purple team is very sloppy today
Pink11 makes a long pass to Pink8
Pink8 passes back to Pink11
turnover ( Purple1, Pink8 )
pass ( Pink11, Pink8 )
pass ( Pink8, Pink11 )
ballstopped
pass ( Pink8, Pink11 )
kick ( Pink11 )
kick ( Pink8)
kick ( Pink11 )
kick ( Pink11 )
kick ( Pink8 )
15
Robocup Sportscaster TraceNatural Language Commentary Meaning Representation
Purple goalie turns the ball over to Pink8
badPass ( Purple1, Pink8 )
Pink11 looks around for a teammate
Pink8 passes the ball to Pink11
Purple team is very sloppy today
Pink11 makes a long pass to Pink8
Pink8 passes back to Pink11
turnover ( Purple1, Pink8 )
pass ( Pink11, Pink8 )
pass ( Pink8, Pink11 )
ballstopped
pass ( Pink8, Pink11 )
kick ( Pink11 )
kick ( Pink8)
kick ( Pink11 )
kick ( Pink11 )
kick ( Pink8 )
16
Robocup Sportscaster TraceNatural Language Commentary Meaning Representation
Purple goalie turns the ball over to Pink8
P6 ( C1, C19 )
Pink11 looks around for a teammate
Pink8 passes the ball to Pink11
Purple team is very sloppy today
Pink11 makes a long pass to Pink8
Pink8 passes back to Pink11
P5 ( C1, C19 )
P2 ( C22, C19 )
P2 ( C19, C22 )
P0
P2 ( C19, C22 )
P1 ( C22 )
P1( C19 )
P1 ( C22 )
P1 ( C22 )
P1 ( C19 )
17
Robocup Data
• Collected human textual commentary for the 4 Robocup championship games from 2001-2004.– Avg # events/game = 2,613– Avg # sentences/game = 509
• Each sentence matched to all events within previous 5 seconds.– Avg # MRs/sentence = 2.5 (min 1, max 12)
• Manually annotated with correct matchings of sentences to MRs (for evaluation purposes only).
18
Overview
• Sportscasting task
• Tactical generation
• Strategic generation
• Human evaluation
19
Tactical Generation
• Learn how to generate NL from MR
• Example:
• Two steps1. Disambiguate the training data
2. Learn a language generator
Pass(Pink2, Pink3) “Pink2 kicks the ball to Pink3”
20
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( Purple5, Purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
21
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( Purple5, Purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
Semantic Parser Learner
Initial SemanticParser
22
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
Initial SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink2 )
Kick ( pink5 )
23
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink2 )
Kick ( pink5 )
Semantic Parser Learner
24
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Semantic Parser Learner
Turnover ( purple7 , pink2 )
25
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Semantic Parser Learner
Turnover ( purple7 , pink2 )
26
System Overview
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Semantic Parser Learner
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
27
Semantic Parser Learners
• Learn a function from NL to MR
NL: “Purple3 passes the ball to Purple5”
MR: Pass ( Purple3, Purple5 )
Semantic Parsing (NL MR)
Tactical Generation (MR NL)
•We experiment with two semantic parser learners–WASP (Wong & Mooney, 2006; 2007)–KRISP (Kate & Mooney, 2006)
28
WASP: Word Alignment-based Semantic Parsing
• Uses statistical machine translation techniques– Synchronous context-free grammars (SCFG)
(Wu, 1997; Melamed, 2004; Chiang, 2005)– Word alignments (Brown et al., 1993; Och &
Ney, 2003)
• Capable of both semantic parsing and tactical generation
29
KRISP: Kernel-based Robust Interpretation by Semantic Parsing
• Productions of MR language are treated like semantic concepts
• SVM classifier is trained for each production with string subsequence kernel
• These classifiers are used to compositionally build MRs of the sentences
• More resistant to noisy supervision but incapable of tactical generation
30
Matching
• Ability to find correct NL/MR pair• 4 Robocup championship games from 2001-2004.
– Avg # events/game = 2,613– Avg # sentences/game = 509
• Leave-one-game-out cross-validation• Metric:
– Precision: % of system’s annotations that are correct– Recall: % of gold-standard annotations correctly
produced– F-measure: Harmonic mean of precision and recall
31
Systems
Learner
KRISPER
(Kate & Mooney, 2007)
KRISP
WASPER WASP
WASPER-GEN WASP’s language generator
32
KRISPER and WASPER
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Semantic Parser Learner
(KRISP/WASP)
Turnover ( purple7 , pink2 )
33
Systems
Learner
KRISPER
(Kate & Mooney, 2007)
KRISP
WASPER WASP
WASPER-GEN WASP’s language generator
34
WASPER-GEN
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
TacticalGenerator
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Tactical Generator Learner(WASP)
Turnover ( purple7 , pink2 )
35
Matching Results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Average results on leave-one-game-out cross-validation
F-m
easu
re randomKRISPERWASPERWASPER-GEN
36
Overview
• Sportscasting task
• Tactical generation
• Strategic generation
• Human evaluation
37
Strategic Generation
• Generation requires not only knowing how to say something (tactical generation) but also what to say (strategic generation).
• For automated sportscasting, one must be able to effectively choose which events to describe.
38
Example of Strategic Generation
pass ( purple7 , purple6 )
ballstopped
kick ( purple6 )
pass ( purple6 , purple2 )
ballstopped
kick ( purple2 )
pass ( purple2 , purple3 )
kick ( purple3 )
badPass ( purple3 , pink9 )
turnover ( purple3 , pink9 )
39
Example of Strategic Generation
pass ( purple7 , purple6 )
ballstopped
kick ( purple6 )
pass ( purple6 , purple2 )
ballstopped
kick ( purple2 )
pass ( purple2 , purple3 )
kick ( purple3 )
badPass ( purple3 , pink9 )
turnover ( purple3 , pink9 )
40
Strategic Generation
• For each event type (e.g. pass, kick) estimate the probability that it is described by the sportscaster.
• Requires correct NL/MR matching– Use estimated matching from tactical
generation– Iterative Generation Strategy Learning
41
Iterative Generation Strategy Learning (IGSL)
• Directly estimates the likelihood of an event being commented on
• Self-training iterations to improve estimates
• Uses events not associated with any NL as negative evidence
42
Strategic Generation Performance
• Evaluate how well the system can predict which events a human comments on
• Metric:– Precision: % of system’s annotations that are correct
– Recall: % of gold-standard annotations correctly produced
– F-measure: Harmonic mean of precision and recall
43
Strategic Generation Results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Average results on leave-one-game-out cross-validation
F-m
easu
re
inferred fromWASPinferred fromKRISPERinferred fromWASPERinferred fromWASPER-GENIGSL
inferred fromgold matching
44
Overview
• Sportscasting task
• Tactical generation
• Strategic generation
• Human evaluation
45
• 4 fluent English speakers as judges• 8 commented game clips
– 2 minute clips randomly selected from each of the 4 games
– Each clip commented once by a human, and once by the machine
• Presented in random counter-balanced order• Judges were not told which ones were human or
machine generated
Human Evaluation (Quasi Turing Test)
46
Demo Clip
• Game clip commentated using WASPER-GEN with IGSL, since this gave the best results for generation.
• FreeTTS was used to synthesize speech from textual output.
47
Human Evaluation
ScoreEnglish Fluency
Semantic Correctness
Sportscasting Ability
5 Flawless Always Excellent
4 Good Usually Good
3 Non-native Sometimes Average
2 Disfluent Rarely Bad
1 Gibberish Never Terrible
48
Human Evaluation
CommentatorEnglishFluency
Semantic Correctness
SportscastingAbility
Human 3.94 4.25 3.63
Machine 3.44 3.56 2.94
Difference 0.5 0.69 0.69
ScoreEnglish Fluency
Semantic Correctness
Sportscasting Ability
5 Flawless Always Excellent
4 Good Usually Good
3 Non-native Sometimes Average
2 Disfluent Rarely Bad
1 Gibberish Never Terrible
49
Future Work
• Expand MRs to beyond simple logic formulas• Apply approach to learning situated language in a
computer video-game environment (Gorniak & Roy, 2005)
• Apply approach to captioned images or video using computer vision to extract objects, relations, and events from real perceptual data (Fleischman & Roy, 2007)
50
Conclusion
• Current language learning work uses expensive, unrealistic training data.
• We have developed a language learning system that can learn from language paired with an ambiguous perceptual environment.
• We have evaluated it on the task of learning to sportscast simulated Robocup games.
• The system learns to sportscast almost as well as humans.
51
Backup Slides
52
Systems
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
53
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
Systems
54
KRISPER and WASPER
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
SemanticParser
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Semantic Parser Learner
(KRISP/WASP)
Turnover ( purple7 , pink2 )
55
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
Systems
56
WASPER-GEN
Purple7 loses the ball to Pink2
Sportscaster Robocup Simulator
Ambiguous Training Data
Pink2 kicks the ball to Pink5
Pink5 makes a long pass to Pink8
Pink8 shoots the ball
Turnover ( purple7 , pink2 )
Pass ( pink5 , pink8)
Pass ( purple5, purple7 )
Kick ( pink2 )Pass ( pink2 , pink5 )Kick ( pink5 )
BallstoppedKick ( pink8 )
TacticalGenerator
Purple7 loses the ball to Pink2
Unambiguous Training Data
Pink2 kicks the ball to Pink5Pink5 makes a long pass
to Pink8Pink8 shoots the ball Kick ( pink8 )
Pass ( pink2 , pink5 )
Kick ( pink5 )
Tactical Generator Learner(WASP)
Turnover ( purple7 , pink2 )
57
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
Systems
58
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
Systems
Matching
59
Matching
• 4 Robocup championship games from 2001-2004.– Avg # events/game = 2,613
– Avg # sentences/game = 509
• Leave-one-game-out cross-validation• Metric:
– Precision: % of system’s annotations that are correct
– Recall: % of gold-standard annotations correctly produced
– F-measure: Harmonic mean of precision and recall
60
Matching Results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Average results on leave-one-outexperiments
F-m
easu
re WASPKRISPERWASPERWASPER-GEN
61
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
Systems
62
Disambiguation Learning language generator
WASP Random WASP
KRISPER
(Kate & Mooney, 2007)
KRISP N/A
WASPER WASP WASP
KRISPER-WASP KRISP WASP
WASPER-GEN WASP’s language generator
WASP
WASP with gold matching
N/A WASP
Lower baseline
Upper baseline
SystemsTactical
Generation
63
Tactical Generation
• 4 Robocup championship games from 2001-2004.– Avg # events/game = 2,613– Avg # sentences/game = 509
• Leave-one-game-out cross-validation• NIST score
– Evaluate the quality of machine translations based on matching n-grams
– BLEU metric with some modifications– More weight for rarer n-grams– Less sensitive to translation length
64
Tactical Generation Results
0
1
2
3
4
5
6
Average results on leave-one-out experiments
NIS
T
WASP
WASPER
KRISPER-WASPWASPER-GEN
WASP with goldmatching
65
Parsing Results
010203040
5060708090
Average results on leave-one-out experiments
F-m
easu
re
WASP
KRISPER
WASPER
KRISPER-WASPWASPER-GEN
WASP with goldmatching
66
Matching Results
67
Parse Results
68
Generation Results
69
Strategic Generation Results
70
Grounded Language Learning in Robocup Robocup Simulator
Sportscaster
Simulated Perception
Perceived Facts
Score!!!!Grounded
Language LearnerLanguageGenerator
SemanticParser
SCFG Score!!!!
71
“Mary is on the phone”
???
72 72
“Mary is on the phone”???
73 73
“Mary is on the phone”???
Ironing(Mommy, Shirt)
74 74
“Mary is on the phone”???
Ironing(Mommy, Shirt)
Working(Sister, Computer)
75 75
“Mary is on the phone”???
Ironing(Mommy, Shirt)
Working(Sister, Computer)
Carrying(Daddy, Bag)
76 76
“Mary is on the phone”???
Ironing(Mommy, Shirt)
Working(Sister, Computer)
Carrying(Daddy, Bag)
Talking(Mary, Phone)
Sitting(Mary, Chair)
77
Grounding Language
“Spanish goalkeeper Iker Casillas blocks the ball”
78
Grounding Language
“Spanish goalkeeper Iker Casillas blocks the ball”
79
Grounding Language
Blocks
• WordNet - (v) parry, block, deflect (impede the movement of (an opponent or a ball)) "block an attack“
• Merriam-Webster - intransitive verb: to block an opponent in sports
80
Grounding Language
Blocks
• WordNet - (v) parry, block, deflect (impede the movement of (an opponent or a ball)) "block an attack“
• Merriam-Webster - intransitive verb: to block an opponent in sports
81
Grounding Language
Blocks
82
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
ballstopped
kick ( purple6 )
pass ( purple6 , purple2 )
turnover ( purple3 , pink9 )
kick ( purple2 )
pass ( purple2 , purple3 )
kick ( purple3 )
Natural Language Commentary Meaning Representation
83
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
pass ( purple6 , purple2 )
pass ( purple2 , purple3 )
Natural Language Commentary Meaning Representation
84
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
kick ( purple6 )
kick ( purple2 )
kick ( purple3 )
Natural Language Commentary Meaning Representation
85
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
kick ( purple 3 )
ballstopped
kick ( purple6 )
pass ( purple6 , purple2 )
kick ( purple2 )
turnover ( purple3 , pink9 )
kick ( purple2 )
pass ( purple2 , purple3 )
kick ( purple3 )
kick (purple 3
Natural Language Commentary Meaning Representation
86
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
kick ( purple 3 )
kick ( purple6 )
kick ( purple2 )
kick ( purple2 )
kick ( purple3 )
kick (purple 3
Natural Language Commentary Meaning Representation
87
Robocup Sportscaster Trace
purple6 passes to purple2
purple3 loses the ball to pink9
purple2 makes a short pass to purple3
kick ( purple 3 )
kick ( purple6 )
kick ( purple2 )
kick ( purple2 )
kick ( purple3 )
kick (purple 3 )
Natural Language Commentary Meaning Representation
Negative Evidence