The class P

1

The class P

Zeph Grunschlag

2

Agenda Running time Big-O review Time complexity classes The class P

Converting RAM programs to TM programs with polynomial loss of efficiency Decision problems encoding I/O problems Factoring numbers

Context free languages are in P CYK algorithm

3

Polynomial Time vs.Exponential Time

Experience shows that the dividing line between problems solvable in polynomial time vs. those requiring exponential time is fundamental. Let’s define the running time of TM’s:

DEF: The running time of a Turing machine M is the function f (n) which is the maximum number of transitions that M takes to halt when given an arbitrary input of length n.

NOTE: A crash is considered as halting.

4

Running Time

Q: What the running time of the above?

0

1

rej acc

2

�R

1R

0R

1R 0R

0R1R

5

Running Time

A: Assuming = {0,1}, the running time is T (n)=n+1` as can be seen on any input starting with 1. Even though the TM runs faster on 0{0,1}*, running time is defined in terms of the worst case. Crashes don’t affect this.

0

1

rej acc

2

�R

1R

0R

1R 0R

0R1R

6

Polynomial Time Complexity

DEF: M is of polynomial time complexity if there is a polynomial p(n) for with the running time f (n) of M satisfies:

f (n) p(n) .Problem: Nobody uses Turing machines in

the real world so is this really justified.Resolution: Investigate the efficiency of

Turing machines vs. actual RAM machines:Strong Church-Turing Thesis: Any

computing device can be simulated by TM’s with at worst a polynomial time slow-down.

7

Strong Church-Turing Thesis?

As opposed to the standard Church-Turing thesis for which there is overwhelming evidence, there is a great deal of current research trying to show that the strong version is false. In particular:

Possible Counterexample: Many conjecture that no polynomial time algorithm exists for factoring numbers using a TM. But a polynomial time quantum algorithm exists (Peter Shor). Would mean that quantum computers invalidate Strong C-T thesis!

8

Big-O Notation

Nothing to say here.Assume you learned in Discrete

Math and Data Structures.

9

Big-O Notation

OK… here’s a refresher:DEF: Let f and g be functions with

domain R0 or N and codomain R. If there are constants C and k such

x > k, |f (x )| C |g (x )|i.e., past k, f is less than or equal to a multiple of g, then we write:

f (x ) = O ( g (x ) )

10

Big- and Big-Big- is just the reverse of big-O. I.e.

f (x ) = (g (x )) g (x ) = O (f (x ))So big- says that asymptotically f (x )

dominates g (x ).Big- says that both functions dominate each-

other so are asymptotically equivalent. I.e.f (x ) = (g (x ))

f (x ) = O (g (x )) f (x ) = (g (x ))Synonym for f = (g): “f is of order g ”

11

Useful facts

Any polynomial is big- of its largest term EG: x 4/100000 + 3x 3 + 5x 2 – 9 = (x

4)

The sum of two functions is big-O of the biggest EG: x 4 ln(x ) + x 5 = O (x 5)

Non-zero constants are irrelevent: EG: 17x 4 ln(x ) = O (x 4 ln(x ))

12

Big-O, Big-, Big-. Examples

Q: Order the following from smallest to largest asymptotically. Group together all functions which are big- of each other:

xex xxexxx

xxxxx ,,,13,1

13,1,,ln,sin

xxxxxxxx 2220 lg,)(ln,ln),102)(sin(

13

Big-O, Big-, Big-. ExamplesA:

1.2.3. , (change of base formula)4. 5.6.7.8.9.10.

xe)102)(sin( 20 xxx

x1

xlnx113x2lg

xxxxx 13,,sin

ex

xx ln2)(ln xx

xx

14

Time Complexity ClassesTime complexity classes are the next

languages classes that we’ll study:DEF: Suppose that g (n) is a real

function. The time complexity class TIME(g (n) ) consists of all languages which are decided by some TM with running time O(g (n)). Any such language is said to by of time complexity g (n).

15

Time Complexity

Q: For which g (n) is the language accepted above of time complexity g (n) ?

1. g (n) = n +22. g (n) = n/23. g (n) = n2

4. g (n) = 1

0

1

rej acc

2

�R

0R

1R 0R

0R1R

16

Time Complexity

A: ALL! This particular decider for the language has running time f (n) = n+2 which is big-O of no.’s 1, 2 and 3. But the definition doesn’t say that must stick to any single implementation:

Q: How would you modify the TM to see that accepted language is in TIME(1) ?

0

1

rej acc

2

�R

0R

1R 0R

0R1R

17

Time Complexity

A: Accepted language is 10{0,1}*. So just need to check first two bits as pictured below:

So now we see that the language is accepted by a TM which runs in constant time.

0 2 acc1

�R 0R1R

1R 0R

18

The Class P

P is the class of languages that are decided by a TM of time complexity some polynomial. IE:

DEF:

0

)(TIMEk

kn P

19

From RAM to TM in Polynomial Time

THM 1: Any RAM program which runs in O (f (n)) time can be simulated by a multi-tape TM which runs in O (f (n)3 ) time.

THM 2: Any multi-tape TM which runs in O (g (n)) time can be simulated by a one-tape TM which runs in O (g (n)2 ) time.

COR: Any RAM program which runs in O (f (n)) time can be simulated by a one-tape TM which runs in O (f (n)6 ) time.

Consequently, any algorithm that runs in polynomial time on your desktop, can be made to run in polynomial time on a TM.

20


No proofs. Rather, give brief explanation: A RAM consists of a “goto” style program and a memory made up of arbitrarily many integers called registers and each having an integer address. A single step of a RAM consists of doing something to one of the registers1. After k steps, a RAM may have at most O (k) active registers each containing an integer of length O (k). If we keep these contents on one of the TM tapes, changing the state of the simulated RAM requires reading all O (k) registers each of size O (k), so O (k 2) tape cells. Since k = O (f (n) ), and there are f (n) steps to simulate, g(n) = O (f (n)3 ).

21


On the other hand, recall conversion from a multi-tape to a 1-tape TM. Converted to a multi-track machine first. The multi-track machine needed to sweep back and forth once across whole tape to read the current active cells. So on multitape’s k ’th step, contents are O (k) so sweeping back and forth is O(k ). Since k = O (g (n) ), and there are g (n) steps to simulate, h(n) = O (g (n)2 ).Now go from multitrack to one-tape machine. Recall that all we did was rename alphabet and transitions so running time was identical to that of 1-tape simulator and to that of k-tape TM!

22

RAM’s are Polynomially Equivalent to TM’s

Other direction valid as well: Can simulate any TM in the same asymptotic time complexity in Java.

Conclude: The belonging of a given language to the class P does not depend on the computing paradigm chosen.

Thus pseudocode is useful in analyzing computational complexity.

23

Turning Decision Problems into I/O Problems

Up till now, point of view of course has been to study computability of algorithmic problems by looking at decision problem versions.

However, in practice, decision problems aren’t always interesting.

EG: If you want to break the RSA encryption algorithm knowing that a number N is composite isn’t good enough. Would want to output the prime factors as well!

24

Encoding FACTORING by a Decision Problem

It turns out, that usually I/0 problems can easily be encoded as decision problems which can yield solutions to the original I/O problem with only polynomial loss of efficiency.

EG: Consider I/O problem FACTORING:Input: a number nOutput: a set of prime numbers p1,p2,

…,pt whose product is the number n.

25


Encode FACTORING by the decision problem BOUNDED_FACTOR:

Given: a number pair (n,k)Decide: is there a number d strictly

between 1 and k which divides n ?Q: Which of the following are in

BOUNDED_FACTOR:1. (10,5)2. (7,8)3. (7,7)

26


A: Which of the following are in BOUNDED_FACTOR:

1. (10,5) Yes, because 2 divides 10 and 1<2<5.

2. (7,8) Yes, because 7 divides itself and 1<7<8

3. (7,7) No, because 7 is prime so does not contain any factors between 1 & 7.

Q: How can PRIME be reduced to BOUNDED_FACTOR ?

27


A: Use the co-mapping reduction f : PRIME BOUNDED_FACTOR

defined by f (n) = (n,n).Now let’s see how to turn a solution to

BOUNDED_FACTOR into a factoring algorithm with polynomial loss of efficiency. The idea is to use a binary search to look for a prime factor. Then divide out by the factor and repeat:

28


m = n, i = 0while(m > 1)

i = i + 1 k = 0, l = m + 1while( k < l - 1 )

if( BOUNDED_FACTOR(m,(k+l)/2) )l = (k+l)/2

elsek = (k+l)/2

pi = k // the smallest prime factor of mm = m/k

return (p1, p2 , … , pi)

29


m = n, i = 0while(m > 1)i = i+1 k = 0, l = m+1while( k < l-1 )

if(BOUNDED_FACTOR(m,(k+l)/2)) l = (k+l)/2

else k = (k+l)/2

pi=k //smallest prime factor of mm = m/k


Let’s see how the algorithm works on input n = 12

30

Encoding FACTORING by a Decision Problemm = n, i = 0while(m > 1)i = i+1 k = 0, l = m+1while( k < l-1 )


else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:

m i k lEG n=12

31



else k = (k+l)/2



m i k l12 0

EG n=12

32



else k = (k+l)/2



m i k l12

1

EG n=12

33



else k = (k+l)/2



m i k l12

1

0 13

EG n=12

34



else k = (k+l)/2



m i k l12

1

0

6

EG n=12

35



else k = (k+l)/2



m i k l12

1

0

3

EG n=12

36



else k = (k+l)/2



m i k l12

1

3

1

EG n=12

37



else k = (k+l)/2



m i k l12

1

3

2

EG n=12

38



else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:2

m i k l12

1

3

2

EG n=12

39



else k = (k+l)/2



m i k l

1

3

2

6

EG n=12

40



else k = (k+l)/2



m i k l

2

3

2

6

EG n=12

41



else k = (k+l)/2



m i k l

2

0 7

6

EG n=12

42



else k = (k+l)/2



m i k l

2

0

3

6

EG n=12

43



else k = (k+l)/2



m i k l

2

3

1

6

EG n=12

44



else k = (k+l)/2



m i k l

2

3

2

6

EG n=12

45



else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:2, 2

m i k l

2

3

2

6

EG n=12

46



else k = (k+l)/2



m i k l

2

3

2

3

EG n=12

47



else k = (k+l)/2



m i k l

3

3

2

3

EG n=12

48



else k = (k+l)/2



m i k l

3

0 4

3

EG n=12

49



else k = (k+l)/2



m i k l

3

4

2

3

EG n=12

50



else k = (k+l)/2



m i k l

3

4

3

3

EG n=12

51



else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:2, 2, 3

m i k l

3

4

3

3

EG n=12

52



else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:2, 2, 3

m i k l

3

4

3

1

EG n=12

53



else k = (k+l)/2


return (p1, p2 , … , pi)Prime factors:2, 2, 3 HALT!

m i k l

3

4

3

1

EG n=12

54



else k = (k+l)/2



Running Time:O (# of prime factors)

O (log n)O (log n)O (log n)O (log n)

O ( f (n) )O (log

n)

O (log n)O (log n)O (log n)

O (1)

55


As complexities of nested loops multiply, the time complexity of FACTORING is O (# of prime factors of n · log n · f (n ) · log n )

CLAIM: The number of prime factors of n is no greater than log n (base 2).

Proof. Each prime factor pi 2 so

n = p1·p2·…·pt 2t where t is the number of factors. Take the log of both sides to conclude that logn t. �

Consequently, FACTORING isO ( (log n )3 · f (n ) )

so that if f (n) is a polynomial, then FACTORING is.

56

CFL PLet’s show that every context free language

admits a polynomial time solution for its acceptance problem.

NOTE: Since the complexity of the algorithm is measured in terms of the strings being tested, and not the representation of the context free language, we may assume any form of representing context free languages, as we find convenient! So let’s assume that the language is represented by a Chomsky normal form grammar.

57

CYK Algorithm for Deciding Context Free

Languages

IDEA: For each substring of a given input x, find all variables which can derive the substring. Once these have been found, telling which variables generate x becomes a simple matter of looking at the grammar, since it’s in Chomsky normal form

58


Languages

Q: Consider the grammar G given byS | AB | XBT AB | XBX ATA aB b1. Is x = aaabb in L(G )2. Is x = aaabbb in L(G )

59


LanguagesThe algorithm is “bottom-up” in that we

start with bottom of derivation tree. I’ve inverted everything:

S | AB | XBT AB | XBX ATA aB b

a a a b b

60


Languages1) Write variables for all length 1

substrings

S | AB | XBT AB | XBX AT

A aB b

a a a b b

A A A B B

61



substrings

S | AB | XB

T AB | XB

X ATA aB b

a a a b b

A A A B B

TS,T

62



substrings

S | AB | XBT AB | XB

X ATA aB b

a a a b b

A A A B B

T

X

S,T

63



substrings


a a a b b

A A A B B

T

X

S,T

S,T

64



substrings. X is the only generator!S | AB | XBT AB | XB

X ATA aB b

REJECT!

a a a b b

A A A B B

T

X

S,T

S,T

X

65


LanguagesNow look at aaabbb :


X ATA aB b

a a a b b b

66



substrings.


X AT

A aB b

a a a b b

A A A B B

b

B

67



substrings.

S | AB | XB

T AB | XB

X ATA aB b

a a a b b

A A A B B

S,T

b

B

68



substrings.


X ATA aB b

a a a b b

A A A B B

TX

b

B

S,T

69



substrings.


a a a b b

A A A B B

TX

S,T

b

B

S,T

70



substrings.


X ATA aB b

a a a b b

A A A B B

TX

S,T

b

B

X

S,T

71



substrings.


S is included soaaabbb accepted!

a a a b b

A A A B B

TX

S,T

b

B

X

S,T

S,T

72


LanguagesCan also use a table for same

purpose. end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

1:aaabbb

2:aaabbb

3:aaabbb

4:aaabbb

5:aaabbb

73


Languages1. Variables for length 1 substrings.

end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A

1:aaabbb

A

2:aaabbb

A

3:aaabbb

B

4:aaabbb

B

5:aaabbb

B

74



end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A -

1:aaabbb

A -

2:aaabbb

A S,T

3:aaabbb

B -

4:aaabbb

B -

5:aaabbb

B

75



end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A - -

1:aaabbb

A - X

2:aaabbb

A S,T -

3:aaabbb

B - -

4:aaabbb

B -

5:aaabbb

B

76



end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A - - -

1:aaabbb

A - X S,T

2:aaabbb

A S,T - -

3:aaabbb

B - -

4:aaabbb

B -

5:aaabbb

B

77



end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A - - - X

1:aaabbb

A - X S,T -

2:aaabbb

A S,T - -

3:aaabbb

B - -

4:aaabbb

B -

5:aaabbb

B

78


Languages6. Variables for aaabbb.

ACCEPTED! end atstart at

1: aaabb

b

2: aaabb

b

3: aaabb

b

4: aaabb

b

5: aaabb

b

6: aaabb

b

0:aaabbb

A - - - X S,T

1:aaabbb

A - X S,T -

2:aaabbb

A S,T - -

3:aaabbb

B - -

4:aaabbb

B -

5:aaabbb

B

79

CYK PseudocodeOn input x = x1x2 … xn :for (i = 1 to n) //create middle diagonal for (each var. A)

if(Axi) add A to table[i-1][i]

for (d = 2 to n) // d’th diagonal for (i = 0 to n-d)

for (k = i+1 to i+d-1) for (each var. A) for(each var. B in table[i][k])

for(each var. C in table[k][k+d]) if(ABC) add A to table[i][k+d]

return Stable[0][n] ? ACCEPT : REJECT

80

CYK PseudocodeComplexity

On input x = x1x2 … xn :for (i = 1 to n) //create middle diagonal for (each var. A)





return Stable[0][n] ? ACCEPT : REJECTO (1)

81








O (n)

82








O (n)O (n2)

83








O (n)O (n2)O (n3)

84








O (n)O (n2)O (n3)

O (1)

85








O (n)O (n2)O (n3)

O (1)O (n)

86








O (n)O (n2)O (n3)

O (1)O (n)

O (n3)

87


Therefore, the class of context free languages is contained in TIME(n3) so all context free languages are in P.

NOTE: There is a way of proving this fact, without resorting to Chomsky normal form. Earley’s algorithm parses any CFG in cubic time, and this is the algorithm used in JavaCFG.

The class P

Documents

Transcript of The class P