Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9....
Transcript of Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9....
![Page 1: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/1.jpg)
July 2015
Microsoft Research Faculty Summit
Dan Roth Department of Computer Science University of Illinois at Urbana-Champaign
Natural Language Understanding with
Common Sense Reasoning
![Page 2: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/2.jpg)
Please…
Page 2
![Page 3: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/3.jpg)
Please… Identify units Consider multiple
interpretations and representations Pictures, text,
spell/phonetics Put it all together:
Determine “best” global interpretation
Satisfy expectations Slide; puzzle
Page 2
![Page 4: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/4.jpg)
Comprehension
Dan is flying to Philadelphia this weekend. Penn is organizing a workshop on the Penn Discourse Treebank. Dan is attending the workshop The Workshop is in Philadelphia
Page 3
![Page 5: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/5.jpg)
Comprehension
Dan is flying to Philadelphia this weekend. Penn is organizing a workshop on the Penn Discourse Treebank. Dan is attending the workshop The Workshop is in Philadelphia
Interpretation builds on expectations that rely on knowledge.
Page 3
![Page 6: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/6.jpg)
Comprehension
Dan is flying to Philadelphia this weekend. Penn is organizing a workshop on the Penn Discourse Treebank. Dan is attending the workshop The Workshop is in Philadelphia
Jan is a black Dutch man.
Jan is a black man.
Interpretation builds on expectations that rely on knowledge.
Page 3
![Page 7: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/7.jpg)
Comprehension
Dan is flying to Philadelphia this weekend. Penn is organizing a workshop on the Penn Discourse Treebank. Dan is attending the workshop The Workshop is in Philadelphia
Jan is a black Dutch man.
Jan is a black man.
Interpretation builds on expectations that rely on knowledge.
Jan is a short Dutch man. Jan is a short man.
Page 3
![Page 8: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/8.jpg)
Comprehension
Dan is flying to Philadelphia this weekend. Penn is organizing a workshop on the Penn Discourse Treebank. Dan is attending the workshop The Workshop is in Philadelphia
Jan is a black Dutch man.
Jan is a black man.
Interpretation builds on expectations that rely on knowledge.
Jan is a short Dutch man. Jan is a short man.
Page 3
![Page 9: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/9.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by ….
Page 4
![Page 10: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/10.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
Page 4
![Page 11: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/11.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
Page 4
![Page 12: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/12.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
Page 4
![Page 13: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/13.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
Page 4
![Page 14: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/14.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
Page 4
![Page 15: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/15.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 16: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/16.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 17: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/17.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 18: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/18.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 19: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/19.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 20: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/20.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 21: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/21.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 22: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/22.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
![Page 23: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/23.jpg)
Natural Language Inferences
At least 14 people have been killed in southern Sri Lanka, police say. The telecoms minister was among about 35 injured in the blast site at the town of Akuressa, 160km (100 miles) south of the capital, Colombo. Government officials were attending a function at a mosque to celebrate an Islamic holiday at the time. The defense ministry said the suicide attack was carried out by …. 49 people were hit by a suicide bomber in Akuressa.
visitors
Page 4
This is an Inference Problem
![Page 24: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/24.jpg)
Natural Language Understanding Natural language understanding decisions are global decisions
that require Making (local) predictions driven by different models trained in
different ways, at different times/conditions/scenarios The ability to put these predictions together coherently Knowledge, that guides the decisions so they satisfy our expectations
Page 5
![Page 25: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/25.jpg)
Expectation is a knowledge intensive component Natural Language Understanding
Natural language understanding decisions are global decisions that require Making (local) predictions driven by different models trained in
different ways, at different times/conditions/scenarios The ability to put these predictions together coherently Knowledge, that guides the decisions so they satisfy our expectations
Page 5
![Page 26: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/26.jpg)
Expectation is a knowledge intensive component Natural Language Understanding
Natural language understanding decisions are global decisions that require Making (local) predictions driven by different models trained in
different ways, at different times/conditions/scenarios The ability to put these predictions together coherently Knowledge, that guides the decisions so they satisfy our expectations
Page 5
![Page 27: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/27.jpg)
Expectation is a knowledge intensive component Natural Language Understanding
Natural language understanding decisions are global decisions that require Making (local) predictions driven by different models trained in
different ways, at different times/conditions/scenarios The ability to put these predictions together coherently Knowledge, that guides the decisions so they satisfy our expectations
Natural Language Interpretation is a Common Sense driven Inference Process that is best thought of as a knowledge constrained optimization problem,
done on top of multiple statistically learned models.
Page 5
![Page 28: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/28.jpg)
Expectation is a knowledge intensive component Natural Language Understanding
Natural language understanding decisions are global decisions that require Making (local) predictions driven by different models trained in
different ways, at different times/conditions/scenarios The ability to put these predictions together coherently Knowledge, that guides the decisions so they satisfy our expectations
Natural Language Interpretation is a Common Sense driven Inference Process that is best thought of as a knowledge constrained optimization problem,
done on top of multiple statistically learned models.
Many forms of Inference; a lot boil down to determining best assignment Page 5
![Page 29: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/29.jpg)
Hayes&McCarthy Frame Problem
Quillian Semantic Networks
ConceptNet
Brooks Subsumption
Minsky, Filmore Frames
2000 1990 1980 1970 1960 1950 1940
Bobrow STUDENT
Winograd SHRDLU
Description Logic
Lenant Cyc
McCarthy Formalizing
Commonsense Simon&Newell
General Problem Solver
A Biased View of Common Sense Reasoning
Common Sense Reasoning was formulated traditionally as a “reasoning” process, irrespective of learning and the resulting knowledge representation.
Page 6
![Page 30: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/30.jpg)
Hayes&McCarthy Frame Problem
Quillian Semantic Networks
ConceptNet
Brooks Subsumption
Minsky, Filmore Frames
2000 1990 1980 1970 1960 1950 1940
Bobrow STUDENT
Winograd SHRDLU
Khardon & Roth Learning to
Reason
Description Logic
Lenant Cyc
McCarthy Formalizing
Commonsense Simon&Newell
General Problem Solver
A Biased View of Common Sense Reasoning
Common Sense Reasoning was formulated traditionally as a “reasoning” process, irrespective of learning and the resulting knowledge representation.
Page 6
![Page 31: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/31.jpg)
What is Needed?
Page 7
![Page 32: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/32.jpg)
What is Needed?
Page 7
![Page 33: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/33.jpg)
What is Needed?
Page 7
![Page 34: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/34.jpg)
What is Needed?
Page 7
![Page 35: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/35.jpg)
What is Needed?
Page 7
A computational Framework
Two Examples: Pronoun
Resolution Quantitative
Reasoning
![Page 36: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/36.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
Page 8
![Page 37: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/37.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
Page 8
![Page 38: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/38.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.10
spouse_of 0.05
born_in 0.85
Page 8
![Page 39: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/39.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.10
spouse_of 0.05
born_in 0.85
Page 8
![Page 40: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/40.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Page 8
![Page 41: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/41.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Page 8
![Page 42: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/42.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Page 8
![Page 43: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/43.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Joint inference gives good improvement
Page 8
![Page 44: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/44.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Key Questions: How to guide the global inference? How to learn the model(s)?
Joint inference gives good improvement
Page 8
![Page 45: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/45.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Models could be learned separately/jointly; constraints may come up only at decision time.
Key Questions: How to guide the global inference? How to learn the model(s)?
Joint inference gives good improvement
Page 8
![Page 46: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/46.jpg)
Joint Inference with General Constraint Structure [Roth&Yih’04,07,….] Recognizing Entities and Relations
Dole ’s wife, Elizabeth , is a native of N.C.
E1 E2 E3 R12 R23
other 0.05
per 0.85
loc 0.10
other 0.05
per 0.50
loc 0.45
other 0.10
per 0.60
loc 0.30
irrelevant 0.10
spouse_of 0.05
born_in 0.85
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.05
spouse_of 0.45
born_in 0.50
other 0.05
per 0.85
loc 0.10
other 0.10
per 0.60
loc 0.30
other 0.05
per 0.50
loc 0.45
irrelevant 0.05
spouse_of 0.45
born_in 0.50
irrelevant 0.10
spouse_of 0.05
born_in 0.85
other 0.05
per 0.50
loc 0.45
Models could be learned separately/jointly; constraints may come up only at decision time.
Key Questions: How to guide the global inference? How to learn the model(s)?
Joint inference gives good improvement
Page 8
![Page 47: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/47.jpg)
Constrained Conditional Models
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 48: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/48.jpg)
Constrained Conditional Models
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 49: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/49.jpg)
Constrained Conditional Models
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 50: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/50.jpg)
Constrained Conditional Models
Weight Vector for “local” models Features, classifiers; log-
linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 51: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/51.jpg)
Constrained Conditional Models
Knowledge component: (Soft) constraints
Weight Vector for “local” models Features, classifiers; log-
linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 52: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/52.jpg)
Constrained Conditional Models
Knowledge component: (Soft) constraints
Weight Vector for “local” models
Penalty for violating the constraint.
How far y is from a “legal/expected” assignment
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 53: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/53.jpg)
Constrained Conditional Models
Training: learning the objective function (w, u)
Decouple? Decompose? Force u to model hard constraints?
Knowledge component: (Soft) constraints
Weight Vector for “local” models
Penalty for violating the constraint.
How far y is from a “legal/expected” assignment
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 54: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/54.jpg)
Constrained Conditional Models
Training: learning the objective function (w, u)
Decouple? Decompose? Force u to model hard constraints?
A way to push the learned model to satisfy our output expectations (or expectations from a latent representation)
[CoDL, Chang et. al (07, 12); Posterior Regularization, Ganchev et. al (10); Unified EM (Samdani et. al (12)]
Knowledge component: (Soft) constraints
Weight Vector for “local” models
Penalty for violating the constraint.
How far y is from a “legal/expected” assignment
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 9
![Page 55: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/55.jpg)
Constrained Conditional Models
Training: learning the objective function (w, u)
Decouple? Decompose? Force u to model hard constraints?
A way to push the learned model to satisfy our output expectations (or expectations from a latent representation)
[CoDL, Chang et. al (07, 12); Posterior Regularization, Ganchev et. al (10); Unified EM (Samdani et. al (12)]
Knowledge component: (Soft) constraints
Weight Vector for “local” models
Penalty for violating the constraint.
How far y is from a “legal/expected” assignment
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y) y = argmaxy ∑ 1Á(x,y) wx,y subject to Constraints C(x,y)
Page 9
![Page 56: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/56.jpg)
Constrained Conditional Models
Training: learning the objective function (w, u)
Decouple? Decompose? Force u to model hard constraints?
A way to push the learned model to satisfy our output expectations (or expectations from a latent representation)
[CoDL, Chang et. al (07, 12); Posterior Regularization, Ganchev et. al (10); Unified EM (Samdani et. al (12)]
Knowledge component: (Soft) constraints
Weight Vector for “local” models
Penalty for violating the constraint.
How far y is from a “legal/expected” assignment
Features, classifiers; log-linear models (HMM, CRF) or a combination
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y) y = argmaxy ∑ 1Á(x,y) wx,y subject to Constraints C(x,y)
Any MAP problem w.r.t. any probabilistic model, can be formulated as an ILP [Roth+ 04, Taskar 04]
Page 9
![Page 57: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/57.jpg)
Examples: CCM Formulations y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 58: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/58.jpg)
Examples: CCM Formulations
While Á(x, y) and C(x, y) could be the same; we want C(x, y) to express high level declarative knowledge over the statistical models.
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 59: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/59.jpg)
Examples: CCM Formulations
While Á(x, y) and C(x, y) could be the same; we want C(x, y) to express high level declarative knowledge over the statistical models.
Formulate NLP Problems as ILP problems (inference may be done otherwise) 1. Sequence tagging (HMM/CRF + Global constraints) 2. Sentence Compression (Language Model + Global Constraints)
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 60: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/60.jpg)
Knowledge/Linguistics Constraints Cannot have both A states and B states in an output sequence.
Examples: CCM Formulations
While Á(x, y) and C(x, y) could be the same; we want C(x, y) to express high level declarative knowledge over the statistical models.
Sequential Prediction HMM/CRF based: Argmax ∑ ¸ij xij
Formulate NLP Problems as ILP problems (inference may be done otherwise) 1. Sequence tagging (HMM/CRF + Global constraints) 2. Sentence Compression (Language Model + Global Constraints)
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 61: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/61.jpg)
Knowledge/Linguistics Constraints Cannot have both A states and B states in an output sequence.
Knowledge/Linguistics Constraints If a modifier chosen, include its head If verb is chosen, include its arguments
Examples: CCM Formulations
While Á(x, y) and C(x, y) could be the same; we want C(x, y) to express high level declarative knowledge over the statistical models.
Sequential Prediction HMM/CRF based: Argmax ∑ ¸ij xij
Sentence Compression/Summarization: Language Model based: Argmax ∑ ¸ijk xijk
Formulate NLP Problems as ILP problems (inference may be done otherwise) 1. Sequence tagging (HMM/CRF + Global constraints) 2. Sentence Compression (Language Model + Global Constraints)
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 62: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/62.jpg)
Knowledge/Linguistics Constraints Cannot have both A states and B states in an output sequence.
Knowledge/Linguistics Constraints If a modifier chosen, include its head If verb is chosen, include its arguments
Examples: CCM Formulations
While Á(x, y) and C(x, y) could be the same; we want C(x, y) to express high level declarative knowledge over the statistical models.
Sequential Prediction HMM/CRF based: Argmax ∑ ¸ij xij
Sentence Compression/Summarization: Language Model based: Argmax ∑ ¸ijk xijk
Formulate NLP Problems as ILP problems (inference may be done otherwise) 1. Sequence tagging (HMM/CRF + Global constraints) 2. Sentence Compression (Language Model + Global Constraints)
Constrained Conditional Models Allow: Decouple complexity of the learned model from that of the desired output Learn a simple model (multiple; pipelines); reason with a complex one. Accomplished by incorporating constraints to bias/re-rank global decisions
to satisfy (minimally violate) expectations.
y = argmaxy 2 Y wTÁ(x, y) + uTC(x, y)
Page 10
![Page 63: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/63.jpg)
I. Coreference Resolution (ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He
is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
Page 11
![Page 64: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/64.jpg)
I. Coreference Resolution (ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He
is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
(ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
Page 11
![Page 65: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/65.jpg)
I. Coreference Resolution (ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He
is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
(ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
Page 11
![Page 66: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/66.jpg)
I. Coreference Resolution (ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He
is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous. Big Problem; essential to text understanding; hard. Requires: good learning and inference models & knowledge
(ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in England. He is the same person that you read about in the book, Winnie the Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him. The poem was printed in a magazine for others to read. Mr. Robin then wrote a book. He made up a fairy tale land where Chris lived. His friends were animals. There was a bear called Winnie the Pooh. There was also an owl and a young pig, called a piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made them come to life with his words. The places in the story were all near Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to read about Christopher Robin and his animal friends. Most people don't know he is a real person who is grown now. He has written two books of his own. They tell what it is like to be famous.
Page 11
![Page 67: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/67.jpg)
Recent Advances in Co-reference [Chang, Peng, Samdani, Khashabi]
Latent Left-linking Model (L3M) model [ICML 14]
Joint mention identification & co-reference resolution [CoNLL’15]
Hard Co-reference Problems [NAACL’15]
Page 12
![Page 68: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/68.jpg)
Recent Advances in Co-reference [Chang, Peng, Samdani, Khashabi]
Latent Left-linking Model (L3M) model [ICML 14] A latent variable structured prediction model for discriminative
supervised clustering. Jointly learns a similarity function and performs inference, assuming a latent left linking forest of mentions.
Joint mention identification & co-reference resolution [CoNLL’15]
Hard Co-reference Problems [NAACL’15]
Page 12
![Page 69: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/69.jpg)
Recent Advances in Co-reference [Chang, Peng, Samdani, Khashabi]
Latent Left-linking Model (L3M) model [ICML 14] A latent variable structured prediction model for discriminative
supervised clustering. Jointly learns a similarity function and performs inference, assuming a latent left linking forest of mentions.
Joint mention identification & co-reference resolution [CoNLL’15] Augment the ILP based Inference formulation with “a legitimate
mention” variable, to jointly determine if the mention is legitimate and what to co-ref it with
Hard Co-reference Problems [NAACL’15]
Page 12
![Page 70: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/70.jpg)
Recent Advances in Co-reference [Chang, Peng, Samdani, Khashabi]
Latent Left-linking Model (L3M) model [ICML 14] A latent variable structured prediction model for discriminative
supervised clustering. Jointly learns a similarity function and performs inference, assuming a latent left linking forest of mentions.
Joint mention identification & co-reference resolution [CoNLL’15] Augment the ILP based Inference formulation with “a legitimate
mention” variable, to jointly determine if the mention is legitimate and what to co-ref it with
Hard Co-reference Problems [NAACL’15]
All together, the outcome is the best end-to-end coreference results on CoNLL data and on ACE [CoNLL’15]
Page 12
![Page 71: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/71.jpg)
Recent Advances in Co-reference [Chang, Peng, Samdani, Khashabi]
Latent Left-linking Model (L3M) model [ICML 14] A latent variable structured prediction model for discriminative
supervised clustering. Jointly learns a similarity function and performs inference, assuming a latent left linking forest of mentions.
Joint mention identification & co-reference resolution [CoNLL’15] Augment the ILP based Inference formulation with “a legitimate
mention” variable, to jointly determine if the mention is legitimate and what to co-ref it with
Hard Co-reference Problems [NAACL’15]
All together, the outcome is the best end-to-end coreference results on CoNLL data and on ACE [CoNLL’15]
Page 12
![Page 72: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/72.jpg)
When Tina pressed Joan to the floor she was punished.
When Tina pressed Joan to the floor she was hurt.
When Tina pressed charges against Joan she was jailed.
Pronoun Resolution can be Really Hard
Page 13
![Page 73: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/73.jpg)
When Tina pressed Joan to the floor she was punished.
When Tina pressed Joan to the floor she was hurt.
When Tina pressed charges against Joan she was jailed.
Pronoun Resolution can be Really Hard
Page 13
![Page 74: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/74.jpg)
Co-reference
State-of-the-art co-reference resolution makes random decisions on problems of this type.
Page 14
![Page 75: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/75.jpg)
When Tina pressed Joan to the floor she was punished.
When Tina pressed Joan to the floor she was hurt.
When Tina pressed charges against Joan she was jailed. Requires, among other things, thinking about the structure of the
sentence – who does what to whom
Pronoun Resolution can be Really Hard
Page 15
![Page 76: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/76.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
Page 16
![Page 77: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/77.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
The bee landed on the flower because it had/wanted pollen.
Page 16
![Page 78: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/78.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
The bee landed on the flower because it had/wanted pollen.
John Doe robbed Jim Roy. He was arrested by the police.
Page 16
![Page 79: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/79.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
The bee landed on the flower because it had/wanted pollen.
John Doe robbed Jim Roy. He was arrested by the police. The Subj of “rob” is more likely than the Obj of “rob” to be the Obj of
“arrest”
Page 16
![Page 80: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/80.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
The bee landed on the flower because it had/wanted pollen.
John Doe robbed Jim Roy. He was arrested by the police. The Subj of “rob” is more likely than the Obj of “rob” to be the Obj of
“arrest”
Knowledge representation called “predicate schemas”
Page 16
![Page 81: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/81.jpg)
Hard Co-reference Problems Requires knowledge Acquisition
The bee landed on the flower because it had/wanted pollen.
John Doe robbed Jim Roy. He was arrested by the police. The Subj of “rob” is more likely than the Obj of “rob” to be the Obj of
“arrest”
Requires an inference framework that can make use of this
knowledge
Knowledge representation called “predicate schemas”
Page 16
![Page 82: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/82.jpg)
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 <= 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
Page 17
![Page 83: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/83.jpg)
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 <= 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
Page 17
![Page 84: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/84.jpg)
Variable 𝒚𝒚𝒖𝒖𝒗𝒗 indicates a coreference link uv ILP Formulation of Coreference Resolution
𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv ⋅ 𝑦𝑦𝑢𝑢𝑢𝑢
s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 <= 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
Page 17
![Page 85: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/85.jpg)
Variable 𝒚𝒚𝒖𝒖𝒗𝒗 indicates a coreference link uv ILP Formulation of Coreference Resolution
𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv ⋅ 𝑦𝑦𝑢𝑢𝑢𝑢
s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 <= 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
Best Link Approach: only one of the antecedents u is linked to v
Page 17
![Page 86: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/86.jpg)
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 < 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
Page 18
![Page 87: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/87.jpg)
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 < 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
Page 18
![Page 88: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/88.jpg)
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 < 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
Acquire knowledge; formulated via “Predicate Schemas”.
predicate schemas
Page 18
![Page 89: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/89.jpg)
3.1 1.5 3.1
-1.5 1.2 0.2
𝒗𝒗 𝒖𝒖
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 < 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
Acquire knowledge; formulated via “Predicate Schemas”. Constraints over predicate schemas are instantiated given a new
instance (document) and are incorporated “on-the-fly” into the ILP-based inference formulation to support preferred interpretations.
predicate schemas
Page 18
![Page 90: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/90.jpg)
ILP Formulation of Coreference Resolution 𝑦𝑦 = arg max𝑦𝑦 ∑𝑢𝑢𝑢𝑢 wuv
⋅ 𝑦𝑦𝑢𝑢𝑢𝑢 s.t ∑𝑢𝑢 < 𝑢𝑢 𝑦𝑦𝑢𝑢𝑢𝑢 < 1, ∀v 𝑦𝑦𝑢𝑢𝑢𝑢 ε {0,1}
Acquire knowledge; formulated via “Predicate Schemas”. Constraints over predicate schemas are instantiated given a new
instance (document) and are incorporated “on-the-fly” into the ILP-based inference formulation to support preferred interpretations.
Results in a state-of-the-art coreference that at the same time also handles hard instances at close to 90% Precision.
predicate schemas
Page 18
![Page 91: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/91.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,…
Page 19
![Page 92: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/92.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together.
Page 19
![Page 93: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/93.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together. Understanding implies mapping the text to an arithmetic
expression, or an equation: E = 3 ∑i oi
Page 19
![Page 94: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/94.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together. Understanding implies mapping the text to an arithmetic
expression, or an equation: E = 3 ∑i oi
~~
Page 19
![Page 95: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/95.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together. Understanding implies mapping the text to an arithmetic
expression, or an equation: E = 3 ∑i oi John had 6 books; he wanted to give it to two of his friends. How many will each one get?
~~
Page 19
![Page 96: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/96.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together. Understanding implies mapping the text to an arithmetic
expression, or an equation: E = 3 ∑i oi John had 6 books; he wanted to give it to two of his friends. How many will each one get?
share it with ~~
Page 19
![Page 97: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/97.jpg)
II. Quantities & Quantitative Reasoning
A crucially important natural language understanding task. Election results; Stock Market; Casualties,… The Emmanuel campaign funding totaled three times that of all his opponents put together. Understanding implies mapping the text to an arithmetic
expression, or an equation: E = 3 ∑i oi John had 6 books; he wanted to give it to two of his friends. How many will each one get?
share it with ~~
Page 19
![Page 98: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/98.jpg)
Mapping Text to Expressions
Gwen was organizing her book case making sure each of the shelves had exactly 9 books on it. She has 2 types of books – mystery books and picture books. If she had 3 shelves of mystery books and 5 shelves of picture books, how many books did she have total?
Page 20
![Page 99: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/99.jpg)
Mapping Text to Expressions
Gwen was organizing her book case making sure each of the shelves had exactly 9 books on it. She has 2 types of books – mystery books and picture books. If she had 3 shelves of mystery books and 5 shelves of picture books, how many books did she have total?
[Roy & Roth’15] suggests a solution that involves “parsing” the problem into an expression tree
Page 20
![Page 100: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/100.jpg)
Mapping Text to Expressions
Gwen was organizing her book case making sure each of the shelves had exactly 9 books on it. She has 2 types of books – mystery books and picture books. If she had 3 shelves of mystery books and 5 shelves of picture books, how many books did she have total?
[Roy & Roth’15] suggests a solution that involves “parsing” the problem into an expression tree
Page 20
![Page 101: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/101.jpg)
Mapping Text to Expressions
Gwen was organizing her book case making sure each of the shelves had exactly 9 books on it. She has 2 types of books – mystery books and picture books. If she had 3 shelves of mystery books and 5 shelves of picture books, how many books did she have total?
[Roy & Roth’15] suggests a solution that involves “parsing” the problem into an expression tree
Page 20
![Page 102: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/102.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
Page 21
![Page 103: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/103.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
E* = argmax ∑q R(q) 1q + (q, q’) Pair(q, q’, ¯(q, q’)) 1q,q’
Page 21
![Page 104: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/104.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
E* = argmax ∑q R(q) 1q + (q, q’) Pair(q, q’, ¯(q, q’)) 1q,q’
Score of q being irrelevant to E
Page 21
![Page 105: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/105.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
E* = argmax ∑q R(q) 1q + (q, q’) Pair(q, q’, ¯(q, q’)) 1q,q’
Score of q being irrelevant to E
Score of ¯ being the unique operation between (qi, qj )
Page 21
![Page 106: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/106.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
E* = argmax ∑q R(q) 1q + (q, q’) Pair(q, q’, ¯(q, q’)) 1q,q’ Subject to commonsense constraints.
Legitimacy Positive Answer; Integral Answer ; Range,…
Score of q being irrelevant to E
Score of ¯ being the unique operation between (qi, qj )
Page 21
![Page 107: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/107.jpg)
Inferring the Best Expression Tree
Decomposition: Uniqueness properties of the Τ(E) implies that it is determined by the unique Τ−operation between pairs of relevant quantities.
E* = argmax ∑q R(q) 1q + (q, q’) Pair(q, q’, ¯(q, q’)) 1q,q’ Subject to commonsense constraints.
Legitimacy Positive Answer; Integral Answer ; Range,…
Score of q being irrelevant to E
Expectations developed given a text snippet
Score of ¯ being the unique operation between (qi, qj )
Page 21
Results in a state-of-the-art results on multiple types of arithmetic word problems
![Page 108: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/108.jpg)
Conclusion Natural Language Understanding is a Common Sense Inference problem.
We would gain by thinking in a unified way on Learning, Knowledge
(Representation and Acquisition) and Reasoning.
Provided some recent samples from a research program that addresses Learning, Inference and Knowledge via A constrained optimization framework that guides “best assignment”
inference, with (declarative) output expectations.
Page 22
![Page 109: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/109.jpg)
Conclusion Natural Language Understanding is a Common Sense Inference problem.
We would gain by thinking in a unified way on Learning, Knowledge
(Representation and Acquisition) and Reasoning.
Provided some recent samples from a research program that addresses Learning, Inference and Knowledge via A constrained optimization framework that guides “best assignment”
inference, with (declarative) output expectations.
Page 22
Check out our CCM tutorial tools, demos, LBJava,…
![Page 110: Natural Language Understanding with Common Sense Reasoningdanroth/Talks/Roth-MSR-07... · 2015. 9. 16. · Natural Language Interpretation is a Common Sense driven Inference Process](https://reader035.fdocuments.net/reader035/viewer/2022070219/612f5d811ecc51586943658c/html5/thumbnails/110.jpg)
Conclusion Natural Language Understanding is a Common Sense Inference problem.
We would gain by thinking in a unified way on Learning, Knowledge
(Representation and Acquisition) and Reasoning.
Provided some recent samples from a research program that addresses Learning, Inference and Knowledge via A constrained optimization framework that guides “best assignment”
inference, with (declarative) output expectations.
Thank You!
Page 22
Check out our CCM tutorial tools, demos, LBJava,…