Complexity 7-1 Complexity Andrei Bulatov Complexity of Problems.
Complexity Theory Lecture 4 Lecturer: Moni Naor. Recap Last week: Space Complexity Savitch’s...
-
Upload
megan-haslem -
Category
Documents
-
view
219 -
download
1
Transcript of Complexity Theory Lecture 4 Lecturer: Moni Naor. Recap Last week: Space Complexity Savitch’s...
Recap
Last week: Space Complexity• Savitch’s Theorem: NSPACE(f(n)) µ SPACE(f2(n))
– Collapse of NPSPACE to PSPACE• PSPACE Completeness
– TQBF, Games• Logarithmic space – deterministic and non• Sublogarithmic space • Non-deterministic space is closed under complementation
This week: Probabilistic Space Complexity
The Central Questions of Complexity Theory
LogSpace µ NL µ P µ NP µ PSPACE
Are any of the containments proper?
All we can say: NL ( PSPACESince NL µ Space(log2 n) ( PSPACE from diagonalization
(space hierarchy) Is NP = Co-NP? Is P = NP Å Co-NP ?
Have not seen yet power of:• Oracles• Interaction• Randomization
If P = NP Å Co-NP then factoring is easy
2-SAT2-SAT: given a formula = C1 Æ C2 … Æ Cm where
Ci =( Ç ) where 2 {xj, :xj} and 2 {xk, :xk} for some j and k.
Is there a satisfying assumption for
Let G() be a directed graph with– Nodes: literals – Edge (, iff clause (: Ç ) exists
Claim: is satisfiable iff for no xi are there both a path from xi to :xi and a path from :xi to xi in G()
Corollary: 2-SAT is in NL easy to see 2-unsat in NL and apply Immerman--Szelepcsenyi
Represent the implication the constraint puts
2-SAT is Complete for NL
• Reduce from (un)reachability of DAGs– This is NL-Complete as well
• Given a DAG G, source s and target t construct (G):– For each node x create variable x– For edge (x,y) 2 G: add clause (:x Ç y) – Add clauses (s) and (:t)
Claim: (G) is satisfiable iff there is no path from s to t in G
Log space construction
Algorithm for 2-SAT• Start with arbitrary assignment• While the assignment is not satisfying
– Choose an arbitrary unsatisfied clause Ci =( Ç )– Choose at random from { , and flip the assignment to that
variable – If unsuccessful for many step stop and declare `unsat’
Analysis: let A 2 {0,1}n be any satisfying assignment– With probability at least ½ distance to A is reduced– With probability at most ½ distance to A is increased
0 n
Analysis of AlgorithmDistance can never be larger than nWant to compute how long it takes a pebble to get
with high probability to 0 if it starts at some 0 · i · n Dominated by a walk where – With probability exactly ½ distance to A is reduced– With probability exactly ½ distance to A is increased– When pebble hits 0 it is absorbed– When pebble hits n it is reflected
0 n
Dominated: for all t probability exact process takes more than t steps is at least as the original process
Analysis of Random Walks
Would like to be able to say:Expected time to visit 0For what time period can we say that there high
probability?
Probabilistic Turing MachinesProbabilistic TM: transition function X Q X Q X {left,right}
may be Probabilistic. • A probability distribution on the set of moves
– Accuracy• Alternative view: in addition to input tape, random coins tape
– How do we count it in space bounded computation • head movement is one-way
When is the PTM M considered to recognize a language L:Two sided error: • For all x 2 L we have Pr[M stops with `yes’]>2/3• For all x 2 L we have Pr[M stops with `no’]>2/3One-sided error• For all x 2 L we have Pr[M stops with `yes’]>1/2• For all x 2 L we have Pr[M stops with `no’]=1zero error: M never stops with the wrong answer but might never stop
Want to consider expected consumption of resources
Always stopsAlways stops
Monte CarloVs.
Las Vegas
Random LogSpace (RL)• A probabilistic Turing Machine with worktape size O(log |X|)
– Long enough to count and assure stopping in timeRL: one-sided Error• For all x 2 L we have Pr[M stops with `yes’]>1/2• For all x 2 L we have Pr[M stops with `no’]=1BPL: two-sided Error• For all x 2 L we have Pr[M stops with `yes’]>2/3• For all x 2 L we have Pr[M stops with `no’]>2/3ZPL: No Error but stopping time is only expected polynomial
time
Show: ZPL= RL Å Co-RL
Undirected Connectivity and RL• Given an undirected graph G=(V,E)
– for nodes s and t, is there a path from s to t.– is the graph connected
• Homework: the two versions are log space equivalent via oracle reductions
Idea of algorithm:• Perform random walk for certain amount of
time• If node t is visited declare connected• Ow unconnected
Random walk:If at node u with degree
du
Choose uniformly at random a neighbor and move to it
Markov Chains• A discrete time stochastic process X1, X2, … Xt …
– defined over a set of states S • Finite or countably infinite
– in terms of transition matrix P• Entry Pij is probability next state is j given current state is i: Pr[Xt+1 =j|Xt =i]
– For all i,j 2 S we have 0 · Pij · 1 and j 2 S Pij = 1
• Memoryless Property: Pr[Xt+1 =j|Xt =i]= Pr[Xt+1 =j|Xt =i, Xt-1 =it-1, … X0 =i0]– what matter is where I am, not how I got here
• Example: pebble walk on the lineS ={0,1, …,n} For all 1<i<n we have Pii+1 = ½ and Pii-1 = ½
P00 = 1 and Pnn-1 = 1 all other Pij = 0
0 n
Markov Chains
• Initial state distribution q(0)=(q1(0),q2
(0), … qn(0))
• State probability vector q(t)=(q1(t),q2
(t), … qn(t))
distribution of the chain at time t q(t+1)=q(t)P and q(t)= q(0)Pt
• Stationary distribution: for Markov Chain with transition matrix P is distribution such that = P If a Markov Chain is in a/the stationary distribution at step t it
remains so foreverSteady state behavior
There can be unique, several or no stationary distributionDepending on the initial distribution
Markov Chains• For states i,j 2 S first transition probability rij
(t) is probability that given X0=i first time state j occurs is at step t
rij(t)=Pr[Xt =j Æ Xs ≠j, 1 · s < t| X0 =i]
• Probability that there is such a visit fij=t>0 rij(t)
• Hitting Time: Let hij be the expected number of steps to reach state j starting from state i. If fij <1 then hij = 1
hij=t>0 t rij(t)
converse not necessarily true in infinite chains
If fij =1 then state i is called persistent
Markov Chains• Underlying directed graph of a Markov Chain:
Edge (i,j) iff Pij > 0
• Markov Chain is irreducible if its underlying graph consists of a single strong connected component
• Periodicity of a state i: GCD of the lengths of the path from i back to itself– Bipartite graphs: periodicity 2– State is aperiodic if it has period 1
Fundamental Theorem of Markov chains
For any irreducible, finite and aperiodic Markov Chain:• All states are ergodic (hii < 1) • There is a unique Stationary distribution • For all i 2 S we have
– i > 0
– fij = 1
– hii = 1/i
Famous Markov Chain: PageRank algorithm [Brin and Page 98]
• Good authorities should be pointed by good authorities
• Random walk on the web graph– pick a page at random– with probability α follow a random outgoing link– with probability 1- α jump to a random page
• Rank according to the stationary distribution
nqF
qPRpPR
pq
11
)(
)()(
Random Walks on Graphs
Undirected graph G=(V,E), |E|=m, |V|=n • The transition matrix
Puv = 1/du if (u,v) 2 E and 0 ow.Claim: is G=(V,E) is connected and non-bipartite then for
all v 2 V v =du/2m
Conclusion: huu = 2m/du
How to make the graph non-bipartite? Add self-loops
Cover and Commute time• Commute time Cuv: expected time for a random walk
starting at u to reach v and come backCuv = huv+hvu
• Cover time Cu(G): expected length of a walk starting at u, visiting every node at least once. C(G) =maxu Cu(G).
Lemma: for any edge (u,v): Cuv = huv+hvu · 2m
Consider Markov chain of (directed) edges.Stationary distribution: uniform (each edge+direction) 1/2m
Time to get from u to v and back on edge (v,u): 2m
Not true for non-edges
Upper bound on Cover Time • Theorem: Cu(G)· 2m (n-1) Proof: consider any spanning tree T of G.
There is a traversal of T v0, v1, … v2n-2 where each edge of T is traversed once in each direction.
Consider random walk starting and ending at v0, that traverses each of the edge of T once in each directionUpper bound on walk is upper bound on cover time
Cu(G)· j=02n-3 hvj vj+1
= (u,w) 2 T Cuw · (n-1) 2m
Conclusion: taking 4n3 as the bound in the random walk suffices for constant probability of success
At most m, from Lemma
Universal Traversal Sequences G is a regular graph of degree d• Labeling: the edges neighboring a node are labeled with
{1,2 …d} in an arbitrary manner– Port number
• Can we find a polynomial sized sequence of labelss1, s2, … sk , si 2 {1,2 …d}
such that following it assures visiting all nodes of the graphYes, it exists. Use the probabilistic method.
• Fix a graph an labeling• Compute probability that sequence is not good for graph• Sum over all graphs (nO(dn))
A log space construction puts connectivity in log space
7
4
7
Hot off the press news• Omer Reingold, Undirected ST-Connectivity in
Log-Space,Available: Electronic Colloquium on Computational
Complexity, Report TR04-094
Important Web Resources on Complexity:• ECCC: http://www.eccc.uni-trier.de• Lance Fortnow’s Computational Complexity Web
Log: http://fortnow.com/lance/complog/
What happens in directed graphs
• There are strongly connected directed graphs with an exponential covering time
• Can count to doubly exponential space probabilistically
• So if we do not insist on always stopping may do directed connectivity in random log space (but exponential time)– uninteresting
Probabilistic Variants of P
RP: one-sided Error• For all x 2 L we have Pr[M stops with `yes’]>1/2• For all x 2 L we have Pr[M stops with `no’]=1BPP: two-sided Error• For all x 2 L we have Pr[M stops with `yes’]>2/3• For all x 2 L we have Pr[M stops with `no’]>2/3ZPP: No Error but stopping time is only expected
polynomial time
The Schwartz-Zippel Algorithm/Theorem
Theorem: Let Q(x1, x2, …, xn) 2 F(x1, x2, …, xn) be a non-zero muitivariate polynomial of total degree d. Fix any finite set S µ F and choose r1, r2, …, rn 2R S. Then
Pr[Q(r1, r2, …, rn)=0] · d/|S|
Proof: by induction on n
Useful when Q is not given explicitly and want to test equality to 0Determinant of a matrix
Matching in Bipartite Graphs
• Let G=(V,U,E) be a bipartite graph, |V|=|U|=n Define A to be the n £ n matrix with |E| variables
Aij= xij if (i,j) 2 E and 0 otherwise
Theorem (Edmonds): G has a perfect matching iff det(A)≠0
Proof:det(A)= 2 Sn
sgn() A1,(1) A1,(2)… A1,(n)
Algorithm for deciding whether a matching exists in bipartite graphs
• Fix a prime P larger than 2n• Choose {rij}(i,j) 2 E 2R GF[P]• Compute det(A({rij}(i,j)))• If non-zero declare matching. Ow, no matching
• In general computing det(A) more expensive than running combinatorial algorithm for matching
• Not true in parallel computation