1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l...
![Page 1: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/1.jpg)
1
Top-Down Parsing
![Page 2: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/2.jpg)
2
Outline Review of recursive-descent parsing
When it does not work
Predictive parsing
![Page 3: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/3.jpg)
3
Recursive-Descent Parsing Parsing: given a string of tokens t1 t2 ... tn, find its
parse tree Recursive-descent parsing: Try all the
productions exhaustively» At a given moment the fringe of the parse tree
is: t1 t2 … tk A …» Try all the productions for A: if A ! BC is a
production, rewrite tree to t1 t2 … tk B C … » Backtrack when the tree does not match the
string » Stop when no more non-terminals
![Page 4: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/4.jpg)
4
When Recursive Descent Does Not Work
Consider a production S S a:» In the process of parsing S we try the above
rule» What goes wrong?
A left-recursive grammar has a non-terminal S
S + S for some
Recursive descent does not work in such cases
![Page 5: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/5.jpg)
5
Elimination of Left Recursion Consider the left-recursive grammar S S |
S generates all strings starting with a and followed by a number of
Can rewrite using right-recursion S S’
S’ S’ |
![Page 6: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/6.jpg)
6
More Elimination of Left-Recursion
In general
S S 1 | … | S n | 1 | … | m
All strings derived from S start with one of 1,…,m and continue with several instances of 1,…,n
Rewrite as
S 1 S’ | … | m S’
S’ 1 S’ | … | n S’ |
![Page 7: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/7.jpg)
7
General Left Recursion The grammar
S A | A S is also left-recursive because
S + S
This left-recursion can also be eliminated See [Dragon book, Section 4.3] for general
algorithm.
![Page 8: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/8.jpg)
8
Summary of Recursive Descent
Simple and general parsing strategy» Left-recursion must be eliminated first» … but that can be done automatically
Unpopular because of backtracking» Thought to be too inefficient
In practice, backtracking is eliminated by restricting the grammar
![Page 9: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/9.jpg)
9
Predictive Parsers Like recursive-descent but parser can “predict”
which production to use» By looking at the next few tokens» No backtracking
Predictive parsers accept LL(k) grammars» L means “left-to-right” scan of input» L means “leftmost derivation”» k means “predict based on k tokens of
lookahead” In practice, LL(1) is used
![Page 10: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/10.jpg)
10
LL(1) Languages In recursive-descent, for each non-terminal and
input token there may be a choice of production LL(1) means that for each non-terminal and token
there is only one production Can be specified as a 2D table
» One dimension for current non-terminal to expand
» One dimension for next token» A table entry contains one production
![Page 11: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/11.jpg)
11
Predictive Parsing and Left Factoring
Recall the grammar
E T + E | T
T int | int * T | ( E )
Hard to predict because» For T two productions start with int» For E it is not clear how to predict
A grammar must be left-factored before use for predictive parsing
![Page 12: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/12.jpg)
12
Left-Factoring Example
Recall the grammar
E T + E | T
T int | int * T | ( E )
• Factor out common prefixes of productions E T X
X + E | T ( E ) | int Y
Y * T |
![Page 13: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/13.jpg)
13
LL(1) Parsing Table Example Left-factored grammar
E T X X + E | T ( E ) | int Y Y * T |
The LL(1) parsing table:int * + ( ) $
E T X T X
X + E T int Y ( E )
Y * T
![Page 14: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/14.jpg)
14
LL(1) Parsing Table Example (Cont.)
Consider the [E, int] entry» “When current non-terminal is E and next input
is int, use production E T X» This production can generate an int in the first
place Consider the [Y,+] entry
» “When current non-terminal is Y and current token is +, get rid of Y”
» Y can be followed by + only in a derivation in which Y
![Page 15: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/15.jpg)
15
LL(1) Parsing Tables. Errors Blank entries indicate error situations
» Consider the [E,*] entry» “There is no way to derive a string starting with
* from non-terminal E”
![Page 16: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/16.jpg)
16
Using Parsing Tables Method similar to recursive descent, except
» For each non-terminal S» We look at the next token a» And choose the production shown at [S,a]
We use a stack to keep track of pending non-terminals
We reject when we encounter an error state We accept when we encounter end-of-input
![Page 17: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/17.jpg)
17
LL(1) Parsing Algorithminitialize stack = <S $> and next (pointer to
tokens)repeat case stack of <X, rest> : if T[X,*next] = Y1…Yn
then stack <Y1… Yn rest>; else error (); <t, rest> : if t == *next ++ then stack <rest>; else error ();until stack == < >
![Page 18: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/18.jpg)
18
LL(1) Parsing ExampleStack Input Action
E $ int * int $ T X
T X $ int * int $ int Y
int Y X $ int * int $ terminal
Y X $ * int $ * T
* T X $ * int $ terminal
T X $ int $ int Y
int Y X $ int $ terminal
Y X $ $ X $ $ $ $ ACCEPT
![Page 19: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/19.jpg)
19
Constructing Parsing Tables LL(1) languages are those defined by a parsing
table for the LL(1) algorithm No table entry can be multiply defined
We want to generate parsing tables from CFG
![Page 20: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/20.jpg)
20
Constructing Parsing Tables (Cont.)
If A , where in the line of A do we place ? In the column of b, where b can start a string
derived from * b » We say that b First()
In the column of b, if can reduce to and b can follow an A» S * A b » We say b Follow(A)
![Page 21: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/21.jpg)
21
Computing First SetsDefinition
First(X) = { b | X * b} { | X * }Algorithm sketch:
1. First(b) = { b }
2. First(X) if X is a production
3. First(X) if X A1 … An
and First(Ai) for 1 i n
4. First() First(X) if X A1 … An
and First(Ai) for 1 i n
![Page 22: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/22.jpg)
22
First Sets. Example Recall the grammar
E T X X + E | T ( E ) | int Y Y * T |
First sets
First( ( ) = { ( } First( T ) = {int, ( }
First( ) ) = { ) } First( E ) = {int, ( }
First( int) = { int } First( X ) = {+, } First( + ) = { + } First( Y ) = {*, } First( * ) = { * }
![Page 23: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/23.jpg)
23
Computing Follow Sets Definition:
Follow(X) = { b | S * X b }
Intuition» If X A B then First(B) Follow(A) and
Follow(X) Follow(B)» Also if B * then Follow(X) Follow(A)» If S is the start symbol then $ Follow(S)
![Page 24: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/24.jpg)
24
Computing Follow Sets (Cont.)
Algorithm sketch:
1. $ Follow(S)
2. First() - {} Follow(X)
For each production A X 3. Follow(A) Follow(X)
For each production A X where First()
![Page 25: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/25.jpg)
25
Follow Sets. Example Recall the grammar
E T X X + E | T ( E ) | int Y Y * T |
Follow sets
Follow( + ) = { int, ( } Follow( * ) = { int, ( }
Follow( ( ) = { int, ( } Follow( E ) = {), $}
Follow( X ) = {$, ) } Follow( T ) = {+, ) , $}
Follow( ) ) = {+, ) , $} Follow( Y ) = {+, ) , $}
Follow( int) = {*, +, ) , $}
![Page 26: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/26.jpg)
26
Constructing LL(1) Parsing Tables
Construct a parsing table T for CFG G
For each production A in G do:» For each terminal b First() do
–T[A, b] = » If First(), for each b Follow(A) do
–T[A, b] = » If First() and $ Follow(A) do
–T[A, $] =
![Page 27: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/27.jpg)
27
Notes on LL(1) Parsing Tables
If any entry is multiply defined then G is not LL(1)» If G is ambiguous» If G is left recursive» If G is not left-factored» And in other cases as well
Most programming language grammars are not LL(1)
There are tools that build LL(1) tables
![Page 28: 1 Top-Down Parsing. 2 Outline l Review of recursive-descent parsing l When it does not work l Predictive parsing.](https://reader038.fdocuments.net/reader038/viewer/2022110322/56649d3e5503460f94a17528/html5/thumbnails/28.jpg)
28
Summary For some grammars there is a simple parsing
strategy
» Predictive parsing
Next time: a more powerful parsing strategy