CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and...
Transcript of CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and...
![Page 1: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/1.jpg)
CS626: NLP, Speech and the Web
Pushpak BhattacharyyaCSE Dept., IIT Bombay
Lecture 15, 17: Parsing Ambiguity, Probabilistic Parsing, sample seminar
17th and 24th September, 2012(16th lecture was on UNL by Avishek)
![Page 2: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/2.jpg)
Dealing With Structural Ambiguity
� Multiple parses for a sentence
� The man saw the boy with a telescope.
� The man saw the mountain with a � The man saw the mountain with a telescope.
� The man saw the boy with the ponytail.
At the level of syntax, all these sentences are ambiguous. But semantics can disambiguate 2nd & 3rd sentence.
![Page 3: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/3.jpg)
Prepositional Phrase (PP) Attachment Problem
V – NP1 – P – NP2(Here P means preposition)
NP attaches to NP ?NP2 attaches to NP1 ?
or NP2 attaches to V ?
![Page 4: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/4.jpg)
• Xerox Parsers•XLE Parser (Freely available)•XIP Parser (expensive)
Linguistic Data Consortium(LDC at UPenn)
European Language Resource Association (ELRA)
Linguistic Data Consortium for Indian Languages (LDC-IL)
Europe India
![Page 5: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/5.jpg)
Parse Trees for a Structurally Ambiguous Sentence
Let the grammar be –
S → NP VP
NP → DT N | DT N PPNP → DT N | DT N PP
PP → P NP
VP → V NP PP | V NP
For the sentence,
“I saw a boy with a telescope”
![Page 6: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/6.jpg)
Parse Tree - 1S
NP VP
N V NP
Det N PP
P NP
Det N
I saw
a boy
with
a telescope
![Page 7: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/7.jpg)
Parse Tree -2
S
NP VP
N V NP PP
Det N P NP
Det NI saw
a boy with
a telescope
![Page 8: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/8.jpg)
Parsing Structural Ambiguity
![Page 9: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/9.jpg)
Parsing for Structurally Ambiguous Sentences
� Sentence “I saw a boy with a telescope”
� Grammar:
S � NP VP
NP ART N | ART N PP | PRONNP � ART N | ART N PP | PRON
VP � V NP PP | V NP
ART� a | an | the
N � boy | telescope
PRON � I
V � saw
![Page 10: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/10.jpg)
Ambiguous Parses
� Two possible parses:
� PP attached with Verb (i.e. I used a telescope to see)
� ( S ( NP ( PRON “I” ) ) ( VP ( V “saw” )
( NP ( (ART “a”) ( N “boy”))
( PP (P “with”) (NP ( ART “a” ) ( N ( PP (P “with”) (NP ( ART “a” ) ( N “telescope”)))))
� PP attached with Noun (i.e. boy had a telescope)
� ( S ( NP ( PRON “I” ) ) ( VP ( V “saw” )
( NP ( (ART “a”) ( N “boy”)
(PP (P “with”) (NP ( ART “a” ) ( N “telescope”))))))
![Page 11: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/11.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
![Page 12: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/12.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
![Page 13: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/13.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
![Page 14: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/14.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
![Page 15: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/15.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
![Page 16: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/16.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
![Page 17: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/17.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
![Page 18: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/18.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
![Page 19: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/19.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
![Page 20: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/20.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed
![Page 21: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/21.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
2 ( ( NP VP ) 1 ) − − Use NP � ART N |
ART N PP | PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”, backup
state (b) used
(b) ( ( PRON VP ) 1)
3
B( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed
![Page 22: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/22.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10 ( ( P NP ) 5 ) − −
11 ( ( NP ) 6 ) − Consumed “with”
![Page 23: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/23.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10 ( ( P NP ) 5 ) − −
11 ( ( NP ) 6 ) − Consumed “with”
12 ( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )(b) ( ( PRON ) 6)
−
![Page 24: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/24.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10 ( ( P NP ) 5 ) − −
11 ( ( NP ) 6 ) − Consumed “with”
12 ( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )(b) ( ( PRON ) 6)
−
13 ( ( N ) 7 ) − Consumed “a”
![Page 25: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/25.jpg)
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S � NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10 ( ( P NP ) 5 ) − −
11 ( ( NP ) 6 ) − Consumed “with”
12 ( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )(b) ( ( PRON ) 6)
−
13 ( ( N ) 7 ) − Consumed “a”
14 ( ( − ) 8 ) − Consume “telescope”
Finish Parsing
![Page 26: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/26.jpg)
Top Down Parsing -Observations
� Top down parsing gave us the Verb Attachment Parse Tree (i.e., I used a telescope)telescope)
� To obtain the alternate parse tree, the backup state in step 5 will have to be invoked
� Is there an efficient way to obtain all parses ?
![Page 27: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/27.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
Colour Scheme :
� Blue for Normal Parse
� Green for Verb Attachment Parse
� Purple for Noun Attachment Parse
� Red for Invalid Parse
![Page 28: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/28.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 � PRON12�
S �NP �VPS1?�NP12�VP2?
![Page 29: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/29.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3?
S1?�NP12�VP2? VP2?�V23�NP3?PP??
![Page 30: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/30.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PPS1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5?
![Page 31: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/31.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
NP35�ART34N45�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5?
![Page 32: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/32.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
NP35�ART34N45�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5?
![Page 33: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/33.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5?
![Page 34: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/34.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PPNP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5?
![Page 35: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/35.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PP
NP68�ART67N78�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5?
![Page 36: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/36.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PP
NP68�ART67N78�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5? PP58�P56NP68�
![Page 37: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/37.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PP
NP68�ART67N78�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5? PP58�P56NP68�
NP38�ART34N45PP58�
![Page 38: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/38.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PP
NP68�ART67N78�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5? PP58�P56NP68�
NP38�ART34N45PP58�
VP28�V23NP38�VP28�V23NP35PP58�
![Page 39: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/39.jpg)
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP �ART N �PP
Bottom Up Parse
NP12 � PRON12�
S �NP �VP VP �V �NP PP
VP2?�V23�NP3? NP35 �ART34�N45
NP �ART �N PP
PP5?�P56�NP6?NP35�ART34N45� NP68�ART67�N7?
NP6 �ART �N PP
NP68�ART67N78�
NP3?�ART34N45�PP5?S1?�NP12�VP2? VP2?�V23�NP3?PP?? NP3?�ART34�N45PP5? NP6?�ART67�N78PP8?
VP25�V23NP35�
S15�NP12VP25�
VP2?�V23NP35�PP5? PP58�P56NP68�
NP38�ART34N45PP58�
VP28�V23NP38�VP28�V23NP35PP58�
S18�NP12VP28�
![Page 40: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/40.jpg)
Bottom Up Parsing -Observations
� Both Noun Attachment and Verb Attachment Parses obtained by simply systematically applying the rulessystematically applying the rules
� Numbers in subscript help in verifying the parse and getting chunks from the parse
![Page 41: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/41.jpg)
Exercise
For the sentence,
“The man saw the boy with a telescope” & the grammar given previously, & the grammar given previously, compare the performance of top-down, bottom-up & top-down chart parsing.
![Page 42: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/42.jpg)
Start of Probabilistic Parsing
![Page 43: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/43.jpg)
Example of Sentence labeling: Parsing
[S1[S[S[VP[VBCome][NP[NNPJuly]]]]
[,,]
[CC and]
[S [NP [DT the] [JJ IIT] [NN campus]]
[ [ is][VP [AUX is]
[ADJP [JJ abuzz]
[PP[IN with]
[NP[ADJP [JJ new] [CC and] [ VBG returning]]
[NNS students]]]]]]
[..]]]
![Page 44: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/44.jpg)
Noisy Channel Modeling
Noisy ChannelSource sentence
Target parse
T*= argmax [P(T|S)]T
= argmax [P(T).P(S|T)]T
= argmax [P(T)], since given the parse theT sentence is completely
determined and P(S|T)=1
![Page 45: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/45.jpg)
Corpus
� A collection of text called corpus, is used for collecting various language data
� With annotation: more information, but manual labor intensive
� Practice: label automatically; correct manually
� The famous Brown Corpus contains 1 million tagged words.
� Switchboard: very famous corpora 2400 conversations, 543 speakers, many US dialects, annotated with orthography and phonetics
![Page 46: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/46.jpg)
Discriminative vs. Generative Model
W* = argmax (P(W|SS))W
Compute directly fromP(W|SS)
Compute fromP(W).P(SS|W)
DiscriminativeModel
GenerativeModel
![Page 47: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/47.jpg)
Language Models
� N-grams: sequence of n consecutive words/characters
� Probabilistic / Stochastic Context Free Grammars:Grammars:� Simple probabilistic models capable of handling
recursion
� A CFG with probabilities attached to rules
� Rule probabilities � how likely is it that a particular rewrite rule is used?
![Page 48: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/48.jpg)
PCFGs
� Why PCFGs? � Intuitive probabilistic models for tree-structured
languages
� Algorithms are extensions of HMM algorithms
� Better than the n-gram model for language � Better than the n-gram model for language modeling.
![Page 49: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/49.jpg)
Data driven parsing is tied up with what is seen in the training data
� “Stand right walk left.” Often a message on the airport escalators.
� “Walk left” can come from :-“After climbing , we realized the walk left was still considerable.”
� “Stand right” can come from: -“The stand right though it seemed at that time, does not stand the scrutiny now.”
� Then the given sentence “Stand right walk left” will never be parsed correctly.
![Page 50: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/50.jpg)
Chomski Hierarchy
Properties of the hierarchy:-
� The containment is Type 2 or CFG
Type 1 or CSG
Type 0
� The containment is proper.
� Grammar is not learnable even for the lowest type from positive examples alone.
Type 3 or regular grammar
3210 TypeTypeTypeType ⊃⊃⊃
![Page 51: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/51.jpg)
Rationalism vs Empiricism
� Scientific paper : -“The pendulum swings back”by Ken Church.
� Every 20 years there is a shift from rationalism to empiricism.� AI approaches are of two types:-
� Rational (Theory and model driven e.g. Brill Tagger; humans decide � Rational (Theory and model driven e.g. Brill Tagger; humans decide the templates)
� Empirical (Data driven e.g. HMM)� Is it possible to classify probabilistic CFG into any of these
categories?� Very difficult to answer.� Anything that comes purely from brain can be put into the category of
“rationalism”.� There is no pure rationalism or pure empiricism
![Page 52: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/52.jpg)
Laws of machine learning
� “Learning from vaccuum (zero knowledge) is impossible”.
� “Inductive bias decides what to learn”.� In case of POS tagger , bias is that we assume the
tagger to be a HMM model.
![Page 53: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/53.jpg)
Formal Definition of PCFG
� A PCFG consists of
� A set of terminals {wk}, k = 1,….,V
{wk} = { child, teddy, bear, played…}
� A set of non-terminals {Ni}, i = 1,…,n
{Ni} = { NP, VP, DT…}
� A designated start symbol N1
� A set of rules {Ni → ζj}, where ζj is a sequence of terminals & non-terminalsNP → DT NN
� A corresponding set of rule probabilities
![Page 54: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/54.jpg)
Rule Probabilities
� Rule probabilities are such that
E.g., P( NP → DT NN) = 0.2
P( NP → NN) = 0.5
i
P(N ) 1i ji ζ∀ → =∑
P( NP → NN) = 0.5
P( NP → NP PP) = 0.3
� P( NP → DT NN) = 0.2 � Means 20 % of the training data parses
use the rule NP → DT NN
![Page 55: CS626: NLP, Speech and the Webcs626/cs626-sem1-2012/lecture_slides/c… · CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 15, 17: Parsing Ambiguity,](https://reader036.fdocuments.net/reader036/viewer/2022062916/5ec4942877a3e524403535e7/html5/thumbnails/55.jpg)
Probabilistic Context Free Grammars
� S → NP VP 1.0
� NP → DT NN 0.5
� NP → NNS 0.3
� NP → NP PP 0.2
� DT → the 1.0
� NN → gunman 0.5
� NN → building 0.5
� VBD → sprayed 1.0� NP → NP PP 0.2
� PP → P NP 1.0
� VP → VP PP 0.6
� VP → VBD NP 0.4
� VBD → sprayed 1.0
� NNS → bullets 1.0