Complexity 3: Outline Feasibility...Complexity 3: Feasibility Dusko Pavlovic Introduction Time and...
Transcript of Complexity 3: Outline Feasibility...Complexity 3: Feasibility Dusko Pavlovic Introduction Time and...
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Complexity Theory — Part 3:Feasibility
Dusko Pavlovic
RHULSpring 2012
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Outline
Introduction: Gödel’s letter
Time and space complexity
Abstract complexity
Reductions, hardness, completeness
Projections, provers and verifiers
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Outline
Introduction: Gödel’s letter
Time and space complexity
Abstract complexity
Reductions, hardness, completeness
Projections, provers and verifiers
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Gödel’s letter
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Gödel’s letter
Context: Entscheidungsproblem
Hilbert (1928): Construct a machine Prov : Σ! → Σ! todecide valid formulas:
Prov(F ) ⇐⇒ ∃Π. Π % F
where F is a formula, and Π a proof.
Church (1935), Turing (1936): Entscheidungsproblem isundecidable: a machine Prove may searchforever.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Gödel’s letter
Gödel’s idea
Bounded Entscheidungsproblem: Construct a machineProv : Σ! × N→ Σ!
Prov(F , n) ⇐⇒ ∃Π. Π % F ∧ |Π| ≤ n
where F is a formula, and Π a proof oflength |Π| ≤ n.
Decidable: Construct a TM to try all proofs of length ≤ n.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Gödel’s letter
Gödel’s idea
Complexity of provability: Define
ψ(F , n) =∣∣∣ run of Prov(F , n)
∣∣∣
φ(n) = maxF{n,ψ(F , n)}! worst case
NP: If φ(n) = O(nc), then machines can feasiblyfind proofs.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
Gödel’s letter
Gödel’s idea
NP ={L(M) ⊆ Σ! | M ∈ NTM ∧
∣∣∣M(x)↓∣∣∣ ≤ O(|x |c)
}
(i.e., the languages decided by NTMs in polynomial time)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
SAT ∈ NP
Task
Determine whether a propositional formula asSATisfiable.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
SAT ∈ NP
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
SAT ∈ NP Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
SAT ∈ NP
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NP
SAT ∈ NP Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Outline
Introduction: Gödel’s letter
Time and space complexity
Definitions
Classes
Abstract complexity
Reductions, hardness, completeness
Projections, provers and verifiers
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Time complexity
Idea
" The time requirement of a machine M for the input xis the length of its run⟨ι, inp (x)
⟩−−→$%
⟨q1, τ1
⟩−−→$%
⟨q2, τ2
⟩−−→$%· · · −−→
$%
⟨qk , τk
⟩
︸!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︷︷!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︸time=k
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Time complexity
Idea
" The time requirement of a machine M for the input xis the length of its run⟨ι, inp (x)
⟩−−→$%
⟨q1, τ1
⟩−−→$%
⟨q2, τ2
⟩−−→$%· · · −−→
$%
⟨qk , τk
⟩
︸!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︷︷!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︸time=k
" For nondeterministic M, then the time requirement ismaximized over the possible paths.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Time complexity
Definition
For a program p
" the time requirement is ˜timep : Σ! ⇀ N
˜timep(x) =∨
y∈Σ!
{|r | | T (p, x , y , r)
}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Time complexity
Definition
For a program p
" the time complexity measure is timep : N⇀ N
timep(n) =∨
|x |=n
{|r | | ∃y . T (p, x , y , r)
}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Time complexity is a computable function
A Turing machine that measures time:
timep(n) = do for |x | = nrun U(m, x);count steps;if U(p, x) halts thenstore maximum;
od
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Space complexity
Idea
" The space requirement of a machine M for the inputx is the greatest span that the head reaches
0 nma b b b a a ab a a b baa
︸!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︷︷!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︸space=m+n
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Space complexity
Idea
" The space requirement of a machine M for the inputx is the greatest span that the head reaches
0 nma b b b a a ab a a b baa
︸!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︷︷!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!︸space=m+n
" For nondeterministic M, the space requirement ismaximized over the possible paths.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Space complexity
Definition
For a program p
" the space requirement is spacep : Σ! ⇀ N
˜spacep(x) =∨
y∈Σ!
{r | T (p, x , y , r)
}
where r is the span of the head in the run r .
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Space complexity
Definition
For a program p
" the space complexity measure is spacep : N⇀ N
spacep(n) =∨
|x |=n
{r | ∃y . T (p, x , y , r)
}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Space complexity is a computable function
A Turing machine that measures space:
spacep(n) = do for |x | = nrun U(m, x);record the offsets;if U(p, x) halts thenadd up the offsets;store maximum;
od
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Definition
For any f : N→ N define
DTIME(f ) ={L(M) ⊆ Σ! | M ∈ DTM . time#M$ ≤ f
}
NTIME(f ) ={L(M) ⊆ Σ! | M ∈ NTM . time#M$ ≤ f
}
DSPACE(f ) ={L(M) ⊆ Σ! | M ∈ DTM . space#M$ ≤ f
}
NSPACE(f ) ={L(M) ⊆ Σ! | M ∈ NTM . space#M$ ≤ f
}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Preorder of functions
Notation
f ≤ g ⇐⇒ ∀x . f (x) ≤ g(x)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Two views of time-bounded machines
1. TM that normally reach a decision in ≤ f (n) stepsapproach: try to construct a machine that solves
the problem in time
2. TM with a counter that clocks out after f (n) stepsapproach: • consider all machines that solve the
problem,• find out which ones reach the decisionin time
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Facts.
" obvious:
DTIME(f ) ⊆ NTIME(f ) DTIME(f ) ⊆ NTIME(f )
" "you cannot use more space than time"
DTIME(f ) ⊆ DSPACE(f ) NTIME(f ) ⊆ NSPACE(f )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Proposition 1
NTIME(f ) ⊆ DTIME(2O(f ))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Proposition 2
NTIME(f ) ⊆ DSPACE(f )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Proposition 3
For f ≥ log
NSPACE(f ) ⊆ DTIME(2O(f ))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Proposition 4 (Savich)
For f ≥ log
NSPACE(f ) ⊆ DSPACE(f 2)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Proposition 5 (Immerman-Szelepcsényi)
For f ≥ log
NSPACE(f ) = co-NSPACE(f )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and spaceDefinitions
Classes
Abstractcomplexity
Reducibility
P vs NP
Complexity classes
Prop. 4
Prop. 5
DTIME(f ) DSPACE(f )
NTIME(f )
co-NSPACE(f )
Prop. 1
Prop. 2
Prop. 3
NSPACE(f )
=
2O2O
f>log f>log
f2
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
OutlineIntroduction: Gödel’s letter
Time and space complexity
Abstract complexity
Definition
Classes
The Hierarchy Theorem
Complexity bounds
Reductions, hardness, completeness
Projections, provers and verifiers
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Task
" Compute computational complexity" Machines that measure complexity of machines
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Computer Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Computer so far
Machines
Behaviors
Programs
−(−)
#−${−}Universal: U (#M$ , x) = M(x)
Notation: {n} (x) = U(n, x)
Extensional?: #{n}$ = n and {#M$} = M
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Computer so far
M
Σ∗⊥Σ!
Σ∗
−(−)
#−$
{−}"
U ′
U ∈M
U : Σ∗ × Σ! ⇀ Σ!
U ′ : Σ∗ → Σ∗Σ!
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Convention
For simplicity, henceforth fix the alphabet
Σ = {0, 1,%}
Notation
" N = {0, 1}∗
" N = N ∪ {∞}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Computer so far
M
NN
N
−(−)
#−$
{−}"
U ′
U ∈M
u : N × N⇀ N
u : N→ NN
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Computer with complexity measure
M
NN
N
−(−)
#−$
{−}
NN
u c
U ,C ∈M
u, c : N × N⇀ N
u, c : N→ NN
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity measure
Definition
An (abstract) complexity requirement is a partial functionc : N × N⇀ N such that
" c(#M$ , x)↓ ⇐⇒ M(x)↓
" c(m, x) ≤ k is decidable
" c (#M ◦M ′$ , x) ≤ c (#M$ ,M ′(x)) + c (#M ′$ , x)
" c (#M ×M ′$ , 〈x ! y〉) ≤ c (#M$ , x) + c (#M ′$ , y)
" c (#read$ , x) = |x |
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity measure
Definition (continued)
The (abstract) complexity measure associated with c is afunction c : N→ N
N which maps the program m ∈ N tothe induced partial function
cm : N −→ N
n 7−→∨
|x |=nc(m, x)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Time is a complexity measure
Lemma 1
˜time is a complexity measure with
˜time (#M ◦M ′$ , x) =˜time (#M$ ,M ′(x)) + ˜time (#M ′$ , x)
˜time (#M ×M ′$ , 〈x ! y〉) =
max{ ˜time (#M$ , x) , ˜time (#M ′$ , y)
}
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Space is a complexity measure
Lemma 2
˜space is a complexity requirement with
˜space (#M ◦M ′$ , x) =
max{space (#M$ ,M ′(x)) , space (#M ′$ , x)
}
˜space (#M ×M ′$ , 〈x ! y〉) =
space (#M$ , x) + space (#M ′$ , y)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Nonexamples
" c(m, x) = u(m, x) is not a complexity measurebecause the predicate U(m, x) = k is not decidable.
" c(m, x) = |m| · |x | is not a complexity measurebecause it is defined even when {m} (x) is not.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The additivity induces the polynomial bounds
" If p[q] denotes a program p that calls a procedure qthen
cp[q](n) ≤ cp(n) · cq(n)
follows from the additivity of c, because p can call qat most cp(n) times.
" The exponential growth of complexity occurs wheneach procedure call makes further procedure calls,like in the exhaustive search algorithms.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Abstract complexity classes
Definition
For any complexity measure c define
DCLASSc(f ) ={L(M) ⊆ Σ! | M ∈ DTM . c#M$ ≤ f
}
NCLASSc(f ) ={L(M) ⊆ Σ! | M ∈ NTM . c#M$ ≤ f
}
Pc =∞⋃
k=0DCLASSc(nk )
NPc =∞⋃
k=0NCLASSc(nk )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Concrete complexity classes
Definition
P = Ptime =∞⋃
k=0DTIME(nk )
NP = NPtime =∞⋃
k=0NTIME(nk )
PSPACE = Pspace =∞⋃
k=0DSPACE(nk )
NPSPACE = NPspace =∞⋃
k=0NSPACE(nk )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Concrete complexity classes
Remarks
" Props. 4 and 5 imply
PSPACE = NPSPACE = co-NPSPACE
" The quest forP # NP
is ongoing.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The hierarchy of complexity classes
Recall for RE and REC
For
K(x) =
1 if U(x , x)↓0 otherwise
we provedK ∈ RE \ REC
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The hierarchy of complexity classes
Now for CLASSc(f )
We show that for
Kf (x) =
1 if c(x , x) ≤ f (|x |)0 otherwise
holds
Kf ∈ CLASSc(O(f · logk f )
)\ CLASSc (O(f ))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Proof of Kf $ CLASSc(f )
Define
Kf (x) =
1 if Kf (x) = 0↑ if Kf (x) = 1
and note
Kf(#Kf $
)⇐⇒ c
(#Kf $ , #Kf $
)≤ f(∣∣∣∣#Kf $
∣∣∣∣)
=⇒ c(#Kf $ , #Kf $
)↓
⇐⇒ Kf(#Kf $
)↓
⇐⇒ ¬Kf(#Kf $
)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Proof of Kf $ CLASSc(f )
HenceKf(#Kf $
)=⇒ ¬Kf
(#Kf $
)
¬Kf(#Kf $
)
c(#Kf $ , #Kf $
)> f(∣∣∣∣#Kf $
∣∣∣∣)
Kf $ CLASSc(f )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The Hierarchy Theorem
Theorem
For every f : N→ N and every complexity measure cthere is k ≥ 1 and a language
Kf ∈ CLASSc(O(f · logk f )
)\ CLASSc (O(f ))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The Hierarchy Theorem
Corollary
For every f and c there is some k ≥ 1
" CLASSc (O(f )) " CLASSc(O(f k+1)
)
" CLASSc(O(logk )
)" Pc " CLASSc (O(exp))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
The Hierarchy Theorem
Corollary
For every f and c there is some k ≥ 1
" CLASSc (O(f )) " CLASSc(O(f k+1)
)
" CLASSc(O(logk )
)" Pc " CLASSc (O(exp))
" NCLASSc (O(f )) " NCLASSc(O(f k+1)
)
" NCLASSc(O(logk )
)" NPc " NCLASSc (O(exp))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Question
Which other functions besides polynomials, logarithmsand exponentials are meaningful as complexity bounds?
Answer
" Pathology: Gap theorem
" Health: c-costructible functions = functions for whichc is the universal machine.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Gap Theorem
For every complexity measure c there is a recursivefunction r : N→ N such that
c#M$(x) > r (|x |) =⇒ c#M$(x) > 2r(|x |)
for all M ∈ TM and all but finitely many x .
Hence
CLASSc (r (|x |)) ⊇ CLASSc(2r(|x |)
)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Definition
A function f : N→ N is c-constructible if there is there is1F ∈ DTM such that f (|x |) = c (#F$ , x), for all x ∈ Σ!.
1can always assume multitape
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Lemma
If f : N→ N is c-constructible then for any g : N→ N holds
CLASSc(f ) ⊆ CLASSc(f ) ⇐⇒ f ≤ g
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Corollary
The recursive function r from Gap Theorem is notc-constructible for any c.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Corollary
The recursive function r from Gap Theorem is notc-constructible for any c.
(because CLASSc(2r ) ⊆ CLASSc(r) but not 2r ≤ r )
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
AbstractcomplexityDefinition
Classes
The Hierarchy Theorem
Complexity bounds
Reducibility
P vs NP
Complexity bounds
Comments
" c-constructible functions are just those for which cplays the role of universal TM.
" c-complexity bounds should always bec-constructible
" to avoid Gap Theorem
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Outline
Introduction: Gödel’s letter
Time and space complexity
Abstract complexity
Reductions, hardness, completeness
Reductions
Cook-Levin: NP-completeness
Projections, provers and verifiers
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Reductions
Definition
Let A,B,R ∈ TM. We say that A is reducible to B by Rand write
AR−→ B
if for all inputs x ∈ Σ! holds
" A(x)↓ ⇐⇒ B(R(x))↓
" A(x)↓ =⇒ A(x) = B(R(x))
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Reductions
Definition
Let C ⊆ TM. We say that A is C-reducible to B when
A −→CB ⇐⇒ ∃R ∈ C. A
R−→ B
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Reductions
Definition
A and B are C-equivalent when
A←→CB ⇐⇒ ∃R ∈ C. A
R−→ B ∧ B
R−→ A
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Reductions
Definition
Let A,B ∈ ℘Σ! and C ⊆ ℘Σ!. The notions of reducibilityand equivalence extend by
A −→CB ⇐⇒ ∃A,B ∈ TM ∃C ⊆ TM .
A = L(A) ∧ B = L(B) ∧C = L[C] ∧A −→CB
A←→C
B ⇐⇒ ∃A,B ∈ TM ∃C ⊆ TM .
A = L(A) ∧ B = L(B) ∧C = L[C] ∧A←→CB
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Reductions
Examples
Let D ⊆ TM be any class containing constants andidentities and C a complexity measure. Then
" M −→DU for allM ∈ TM
" F −→DC iff F is a C-constructible function.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Properties of −→C
Proposition 6
(a) preordering
A −→CA and A −→
CB ∧ B −→
CD =⇒ A −→
CD
(b) lower closed in C
A −→CB ∧ B ∈ C =⇒ A ∈ C
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Equivalence
Proposition 7
Any two complexity measures C and D are recursivelyequivalent:
C ←→REC
D
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Definition
Let B ∈ ℘Σ! and L,C ⊆ ℘Σ!. We say that B is L-hardover C
X ∈ L =⇒ X −→CB
We write L −→CB.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Definition
Let B ∈ ℘Σ! and L,C ⊆ ℘Σ!. We say that B isL-complete over C if
" L −→CB
" B ∈ L
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Examples
Let D be any class of languages containing constantsand identities (i.e. the diagonal languages {〈x , x〉}) and Ca complexity class.
" The halting language H = {〈p, x〉 | U(p, x)↓} isRE -complete over D.
" The language consisting of programs in someC-bounded class is complete over D for the family ofC-constructible functions.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Proposition 8
If C ⊆ ℘Σ! is closed under composition, i.e.
M ,N ∈ C =⇒ M ◦ N ∈ C
then(∃B ∈ C. L −→
CB)
=⇒ L ⊆ C
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Corollary
(∃B ∈ P. NP −→
PB)
=⇒ NP = P
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Trouble
If C ⊆ REC is closed under composition and conditionals,i.e.
M ,N ∈ C =⇒ M ◦ N ∈ C∧ (if M then N) ∈ C
then every nonempty B " Σ! is C-hard over C
C −→CB
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Hardness and completeness
Trouble
If C ⊆ REC is closed under composition and conditionals,i.e.
M ,N ∈ C =⇒ M ◦ N ∈ C∧ (if M then N) ∈ C
then every nonempty B " Σ! is C-hard over C
C −→CB
(All main classes are closed under composition and conditionals.)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Moral
Complexity classes
" cannot be meaningfully studied in terms ofreducibility over themselves, but
" over a smaller class, say
L = DSPACE(log)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
Convention
By default
A −→ B means A −→LB
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
NP-completeness
Cook-Levin Theorem
NP −→ SAT
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
NP-completeness
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
ReducibilityReductions
Cook-Levin
P vs NP
NP-completeness Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
OutlineIntroduction: Gödel’s letter
Time and space complexity
Abstract complexity
Reductions, hardness, completeness
Projections, provers and verifiers
Projections
coNP
Summary
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Recall: Projections
Definition
For any family of languages FAM ⊆ ℘(Σ! × Σ!) theprojection family is defined
∃FAM = {L∃ ⊆ Σ! | L ∈ FAM}
where L∃ = {y ∈ Σ! | ∃x . 〈x , y〉 ∈ L}.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Recall: RE = ∃REC
Proof of ∃REC ⊆ RE
M∃(y) = µx . 1 −M(x , y) = 0
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Recall: RE = ∃REC
Proof of ∃REC ⊆ RE
M∃(y) = µx . 1 −M(x , y) = 0
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Recall: RE = ∃REC
Proof of RE ⊆ ∃REC
Kleene’s recursive predicate T : Σ!4 → {0, 1}
U(m, x) = y ⇐⇒ ∃r . T (m, x , y , r)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Polynomial projections
Definition
For any family of languages FAM ⊆ ℘(Σ! × Σ!) theprojection family is defined
∃pFAM = {L∃q ⊆ Σ! | L ∈ FAM ∧ q ∈ Z[n]}
where L∃q = {y ∈ Σ! | ∃x . |x | ≤ q(|y |) ∧ 〈x , y〉 ∈ L}.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
NP = ∃pP
Proof of ∃pP ⊆ NP
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
NP = ∃pP
Proof of NP ⊆ ∃pP
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Interpretation
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Interpretation Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Why is P " NP harder than REC " RE?
RE # REC;
some X ∈ ∃REC \ REC
NP # P;
some X ∈ ∃P \ P
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Why is P " NP harder than REC " RE?
K ∈ ∃REC \ REC" does not show thatfinding witnesses ishard
" shows the decider ofhalting may not halt
" ∃ says that there isa halting run
X ∈ ∃P \ P" must show thatfinding witnesses ishard
" all NP deciders halteventually
" ∃ says that theyhalt in poly time
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Why is P " NP harder than REC " RE?
K ∈ ∃REC \ REC" does not show thatfinding witnesses ishard
" shows the decider ofhalting may not halt
" ∃ says that there isa halting run
X ∈ ∃P \ P" must show thatfinding witnesses ishard
" all NP deciders arestopped in poly time
" ∃ says that thereis a deciding run
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
RE = ∃REC
B ∈ RE;
there is A ∈ REC.(y ∈ B ⇐⇒ ∃x ∈ Σ∗. 〈x , y〉 ∈ A
)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
coRE = ∀REC
B ∈ coRE;
there is A ∈ REC.(y ∈ B ⇐⇒ ∀x ∈ Σ∗. 〈x , y〉 ∈ A
)
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
NP = ∃pP
B ∈ coNP;
there are A, q ∈ P.
y ∈ B ⇐⇒ ∃x ∈ Σ∗.
|x | ≤ q(|y |)⇓
〈x , y〉 ∈ A
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
coNP = ∀pP
B ∈ coNP;
there are A, q ∈ P.
y ∈ B ⇐⇒ ∀x ∈ Σ∗.
|x | ≤ q(|y |)⇓
〈x , y〉 ∈ A
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Interpretation Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
Interpretation
" NP = B = set of satisfiable properties" if 〈x , y〉 ∈ A means "program x satisfies y"" then for every property y ∈ B = ∃pA there issome program x in A that satisfies y
" NP # P because it is hard to find it.
" coNP = B = set of satisfied properties" if 〈x , y〉 ∈ A means "program x satisfies y"" then for every property y ∈ B = ∃pAall programs x in A satisfy y
" coNP # P because it is hard to prove that
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
NP ∩ coNP
" although RE ∩ coRE = REC" if y ∈ L ∈ RE then ML(y) will halt and prove y ∈ L" if y $ L ∈ coRE then ML(y) will halt and prove y $ L.
" it is believed that NP ∩ coNP & P, because having" a family of programs A where y ∈ B ⇐⇒ some xsatisfies it with 〈x , y〉 ∈ A, and
" a family of programs C where y ∈ B ⇐⇒ every zsatisfies it with 〈z , y〉 ∈ C
does not yield feasible way to eliminate search forbuilding x and eliminating z.
Complexity 3:Feasibility
Dusko Pavlovic
Introduction
Time and space
Abstractcomplexity
Reducibility
P vs NPProjections
Hierarchy
Summary
The Inner Eggs