A Hierarchy of Formal Languages and Automata
description
Transcript of A Hierarchy of Formal Languages and Automata
![Page 1: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/1.jpg)
CS 3240 – Chapter 11
![Page 2: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/2.jpg)
They may not halt on every possible input! And not just because the creator of a specific
TM was a doofus
This is related to the major mathematical/computational discovery of the 20th century! There are propositions that cannot be decided
(“proven”)
![Page 3: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/3.jpg)
A question is decidable if there is a TM that always halts and answers “yes” or “no” for each possible input The TM therefore constitutes an algorithm
A language is decidable if there is a TM that always halts and answers “accept” or “reject” whenever an input string is in the language or not aka “Turing acceptable”
![Page 4: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/4.jpg)
A function is computable if there is a TM that always halts with the appropriate output for each possible input in the function domain aka “Total function”
![Page 5: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/5.jpg)
Let g(x,y) be some computable function Let f(x) =
the smallest p where g(x, p) = 1, or0, if such a p does not exist
“Pseudo-algorithm” for f(x):m = 0;while (g(x, m) != 1)
++m;cout << m;
If there is no m for a given g and x, then we hang!
![Page 6: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/6.jpg)
3 possibilities in general when a TM processes an input string: Accepts (goes to an accepting halt state) Rejects (e.g., crashes, or gives a “no”
answer) Hangs (infinite loop)
![Page 7: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/7.jpg)
A language for which there is a TM that always halts for and accepts strings in the language is recognizable “It knows one when it sees one” :-) But it may hang on strings not in the
language
![Page 8: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/8.jpg)
A function that is defined for only some of its domain elements is a partial function It may hang (or “blow up”) on some inputs
e.g., divide by zero This is the computational analogue to a
recognizable language you’ll always get an answer with a valid input
it may hang on invalid input
![Page 9: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/9.jpg)
Language Machine Grammar
Regular Finite Automaton Regular Expression,Regular Grammar
Context-Free Pushdown Automaton
Context-Free Grammar
Recursively Enumerable
Turing Machine Unrestricted Phrase-Structure Grammar
9CS 3240 - Introduction
![Page 10: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/10.jpg)
A language L over the alphabet is called recursively enumerable (aka “recognizable”) if there is a TM T that accepts every word in L and either rejects (crashes) or loops forever for every word in the language L', the complement of L:
accept(T) = L reject(T) + loop(T) = L'
recursively enumerable (r.e.) = recognizable
![Page 11: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/11.jpg)
We just saw one (slide 5) Let g(x,y) be some computable function Let f(x) =
the smallest p where g(x, p) = 1, or0, if such a p does not exist
Let Lg be the set of functions, f, corresponding to all computable functions g, as explained above Such functions can be encoded as strings, and are
therefore countable
Lg can then be seen as a language (a set of strings)
![Page 12: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/12.jpg)
A language L over the alphabet is called recursive (aka “decidable”) if there is a TM T that accepts every word in L and rejects every word in L'; that is,
accept(T) = L
reject(T) = L'
A TM that accepts a recursive set is a model of an algorithm. It always halts.
recursive = decidable
![Page 13: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/13.jpg)
There exist languages that are r.e. but not recursive We just saw one
There exist languages that aren't even r.e.! (You'll see one soon)
All are “contrived” Languages generated by grammars are
r.e. or “better”
![Page 14: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/14.jpg)
All r.e. languages are closed under union, intersection, concatenation, and Kleene* Everything but complement!
Recursive languages are also closed under complement
Also: If L and L' are r.e., then L is recursive
Here come the Proofs…
![Page 15: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/15.jpg)
Let M be a machine that decides a recursive language, L
Form the machine M' by inverting the acceptability output of M Goes to a reject state instead
Then M' decides L' So L' is recursive
![Page 16: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/16.jpg)
Suppose L and L' are both r.e. Let M recognize L, and M' recognize L'
M may hang on elements of L', but M' doesn't Form a new machine, M* that calls M and
M' in parallel (non-deterministically) If M accepts w, so does M*
If M' accepts w, reject w There are no other possibilities! (No hanging) Therefore, L is decidable/recursive, by
definition
![Page 17: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/17.jpg)
TMs can recognize/accept strings from certain languages and/or compute functions
If there is a TM, M, that accepts a language, L, and M always halts, then L is recursive
If there is no such M for L, but there is instead a machine M that accepts every string in L, but M may hang on strings not in L, then L is recursively enumerable
![Page 18: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/18.jpg)
The complement of a recursive language is recursive In fact, recursive languages are closed
under all operations, like regular languages are
r.e. languages are closed under intersection
The complement of a r.e. language may not be r.e. But if it is, then both languages are
actually recursive!
![Page 19: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/19.jpg)
A Timely Interlude
![Page 20: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/20.jpg)
The real numbers in (0,1) are uncountable They cannot be mapped in a 1-to-1
fashion to the counting numbers Proof:
Assume they can be: r0, r1, r2, … Arrange their digits in a table a[ ][ ]▪ each row, a[i], contains the digits or ri
The diagonal sequence (a[n][n]+1) mod 10, representing a valid real number, is not in the table! Contradiction!
![Page 21: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/21.jpg)
The power set of the natural numbers, N the set of all subsets (2N)
Suppose it is countable The we can enumerate the sets: p0, p1, p2, …
Now consider the set T = {i | i ∉ f(i) = pi } Certainly T is a set of integers, so T ∈ P(N)▪ Call it pk
Question: Is k in T = pk?
![Page 22: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/22.jpg)
The power set of any countably infinite set is uncountable
Much bigger than a countable set! The number of countable sets is
negligible compared to the number of uncountable sets
![Page 23: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/23.jpg)
Well?
![Page 24: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/24.jpg)
A language over an alphabet Σ is a subset of Σ*
the latter being an infinite setThe set of all languages over Σ is
therefore the power set of Σ* (2∑*) which we just showed is uncountable
So… the number of languages over any finite alphabet is uncountable
![Page 25: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/25.jpg)
The # of TMs are countable, but the # of languages is not
Therefore, some languages cannot be recognized by a TM There aren't enough TMs to go around! Just like there are more reals than
integersSo, non-r.e. languages must exist!
![Page 26: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/26.jpg)
Just take the complement of any r.e. language that is not recursive
Example (page 279): Consider all TMs, Mi with alphabet Σ={a}.
Let X = {ai: ai ∈ L(Mi)}▪ This is r.e., but not recursive▪ Because we can construct a TM that carries out
the computation, but it may not halt when ai ∉ L(Mi)
Then X’ must be non-r.e.!
![Page 27: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/27.jpg)
Suppose X ’ is r.e. Then there is a TM, Mk, that recognizes it
Now ask the question, is ak ∈ X’ ? If it is, this means ak ∉ L(Mk) = X’, by
definition Contradiction!
If it isn’t, then this means that ak ∈ L(Mk) = X’ Again, by definition Again, a contradiction!
⇒ X ’ is not r.e.
![Page 28: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/28.jpg)
What we have just described is the membership problem: “Given a r.e. language, L, and a string,
w, is w in L?”We have just shown that the
membership problem is, in general, undecidable
![Page 29: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/29.jpg)
Language Machine Grammar
Regular Finite Automaton Regular Expression,Regular Grammar
Context-Free Pushdown Automaton
Context-Free Grammar
Recursively Enumerable
Turing Machine Unrestricted Grammar
29CS 3240 - Introduction
![Page 30: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/30.jpg)
Left-hand side of the rule is a concatenation of one or more symbols There must be at least one variable on
the left▪ λ is not allowed on the left
Any string is allowed on the right, including λ
aka “Type 0” Grammar
![Page 31: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/31.jpg)
S → aAbc | abc | λAb → bAAc → BbccbB → BbaB → aa | aaA
S ⇒aAbc ⇒abAc ⇒abBbcc ⇒aBbbcc ⇒aabbcc
An A is created as the left-most variable. It travels to the right until it finds a 'c', then creates a new b and c, and becomes a B. The B moves back to create the extra needed a. The last rule allows the option to do it all over again (by introducing another A). This is similar to what the Turing machine for the same language does, except the TM marks instead of generates the letters.
![Page 32: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/32.jpg)
We have already seen CFGs for this
This unrestricted grammar: Introduces X’s and
Y’s together Allows moving X’s
and Y’s arbitrarily by swapping them
S → XYS | λXY → YXYX → XYX → aY → b
![Page 33: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/33.jpg)
Cannot use the decidability algorithms for CFGs e.g., CYK algorithm does not apply
No “Normal Form”Non-null productions may create
shorter stringsTerminals can disappear!
![Page 34: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/34.jpg)
For every r.e. language, there is an unrestricted grammar that generates it
For every unrestricted grammar, there is a TM that recognizes its language It may or may not decide it
We will not prove this but the book does
![Page 35: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/35.jpg)
Like Unrestricted Grammars except: Right-hand side must be no shorter than left-
hand side▪ So strings never shrink
Hence the name “non-contracting” or “monotonic” Cannot contain λ anywhere
Context-sensitive languages don't need unlimited memory Since intermediate sentential forms never shrink,
the largest memory requirement is proportional to |w|
Accepting machine: Linear Bounded Automaton
![Page 36: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/36.jpg)
S → aAbc | abcAb → bAAc → BbccbB → BbaB → aa | aaA
![Page 37: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/37.jpg)
Every context-sensitive language is recursive there is a TM that accepts it (i.e., always
halts)But not all recursive languages are
context-sensitiveSo Context Sensitive languages form a
proper subset of Recursive Languages Just like Recursive languages are a proper
subset of Recursively enumerable languages
![Page 38: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/38.jpg)
Type 3 grammar
Type 2 grammar
Type 1 grammar
Type 0 grammar
![Page 39: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/39.jpg)
![Page 40: A Hierarchy of Formal Languages and Automata](https://reader036.fdocuments.net/reader036/viewer/2022081503/56813c09550346895da56ef2/html5/thumbnails/40.jpg)
Language Grammar Machine
Recursively Enumerable(“Turing Recognizable”)
Unrestricted Phrase Structure (Type 0)
TM (that may not halt)
Recursive(“Turing Decidable”)
" TM (that always halts)
Context-Sensitive Context Sensitive (Type 1)(monotonic/non-contracting)
Linear-bounded automata(bounded TM)
Context Free Context Free (Type 2) PDA
Deterministic Context Free
" DPDA
Regular Regular (Type 3)(Right-linear or Left-linear)
FA