Lecture

Today• Problem set 9 out (due next Thursday)• Topics:

– Complexity Theory– Optimization versus Decision Problems– P and NP– Efficient Verification– NP- Completeness– Section: Network Flow

So Far...• Algorithmic Techniques : Divide and Conquer,

Greedy, Dynamic Programming, DFS)• Data Structures: Trees, Heaps, Graphs

• Practice: many problems even if try every trick in the book, nothing seems to work

• Some Problems: Intrinsic Difficulty --Domain of complexity Theory -- 6.045

Complexity Theory

• Halting problem: Given a computer program and an input, does the program halt on that input? -- Unsolvable

• Today: Look at problems which are Solvable but not efficiently

The Clique ProblemDef: Let G=(V,E) be an undirected graph.A clique of G is a subset V’ of the vertices s.t. for all u,v in V’, (u,v) is an edge.I.e clique is a complete sub-graph of G

Optimization ProblemsOptimization: minimize of maximize some valueOptimization Problem for Clique: Given graph G, find size of the

largest clique.How to solve this problem?

– For j = |V| down-to 1try all subsets of j vertices, if find clique, output j.

Time: 2O(|V|) VERY SLOW!

– Is there a “tractable” (efficient) algorithm?– What do you mean by tractable (efficient)?

Traveling Salesman problem

• Def: Given n cities and cost cij for traveling between every two cities, what is the cost of the cheapest way to visit every city exactly once and return to starting city.

• How to solve this problem?Try all possible orderings of cities, and

evaluate each orderings cost, output the cheapest cost.

Time: n! (for n cities )Is there a “tractable” (efficient) algorithm?

Q:What do we mean by efficient:A: Polynomial Time Algorithms

Defn: An algorithm runs in polynomial time if there exists k s.t. the running time is O(nk) on any problem instance of size n.

Q1: What do we mean by “size n”?Q2: Why is this a reasonable choice for “efficient”

EncodingA: Any problem instance can be encoded as a sequence of 0’s

and 1’sAssume all numbers in encoding are in binary?

Size of problem instance = length of encodingMultiple arguments are coded into 1 string.

Choice of the encoding doesn’t matter as long as can compute one from another in polynomial time

Example : FactoringGiven number n, determine m that divides n• Suppose n is encoded in unary: 1111111…1then check all numbers m=2,…,sqrt(n), takes O(sqrt(n) ) = O(sqrt(L)) divisions• Suppose n is encoded in binary, L=log n,same procedure takes O(2L/2) divisions

L=n

Pitfalls• Some functions don’t look like polynomials but are

bounded above by one. E.g n log n<O(n2)• Sometime what you think is polynomial isn’t.

Example: Take as input <G,k> where G is a graph of n nodes, and k an integer. An algorithm of running time o(nk) for this input, is NOT polynomial. Why? Because k is an input to the problem ---The constant in the polynomial should be

independent of the input size.

Justification of ‘Efficient = Polynomial Time”

• Exponential time scales badly -- 2n for n=1000 is larger than number of atoms in universe

• Want our theory of efficient algorithms to be model-independent (I.e hold for any reasonable model of computer). The polynomial-time definition of “efficiency” ensures this

• Bulk of problems that are solvable in polynomial time that are of practical interest, are bounded by small polynomial: MST, shortest path, numerical operation.

Polynomial Time AlgorithmsMost problems so far have poly time algorithmsQ: Does Clique have a poly time algorithm?Q: Does TSP have a poly time algorithm?

A: No one knows. TOC community would be surprised if answer is positive.

Decision Problems: Y/N output Decision problem for clique: Given <G,k>, G=(V,E) is an undirected graph, k is an integer, does G

have a clique of size k?For purely technical reason in complexity theory easier to deal with

decision problems.Comment: Even though seems less natural, as complexity theory task is to show that certain

problems cannot be solved efficiently, showing it for decision version implies it for corresponding optimization problem.

Note: A polynomial time procedure for the clique decision problem should run in time poly(m,n,log k)

where n=|V|, m=|E|

Theorem (for clique):optimization problem solvable in polynomial time iff decision problem solvable in polynomial time

Proof (for clique): (opt dec) Find max clique size, compare with k(dec opt) Binary search.

Natural Question: true for other problems?Yes…when the thing to be optimized has a polynomial # of possible

values in the zize o fthe input instance

Search vs. Decision• Search problem for clique: Given <G,k>, either find a clique of size k, or say

none exists.• Theorem:Search Problem is solvable in polynomial time iff Decision problem is

solved in polynomial time• Proof: Suppose we have a magic-algorithm MB s.t MB(<G,k>) = yes iff there

exists a clique of size k. Then, on <G,k> :

• Run MB(<G,k>), if MB(<G,k>) = no, output no,

• else Order edges of G from 1,…,m, initialize G’= G, I=1 1. run MB(<G’,k> ) where G’ = G- edge I 2. If MB(<G’,k>) = no, then G’=G + edge I, I= I+1,

else I=I +1. 3. If I>n, output edge set of G’, else goto 1

Formal Languages• Def: a formal language is a set of binary strings.• Every decision problem can be viewed as a language

CLIQUE = { <G,k>: G has a clique of size k} (I.e binary encodings of all graphs G that have clique of size k)

TSP = {<G,edge costs,k>: G has a TS Tour of cost < k}• In general, for decision problem D, the corresponding language

is L={x in Sn: D(x)=1}where Sn is the encoding alphabet. (e.g. for binary encoding S={0,1}

Decision Problem: Given <G,u,v,k> where G is a graph, u, v vertices, and integer k>0, is there a path from u to v in G of length < k?

Corresponding language:PATH = {<G,u,v,k>: G is a graph, u,v vertices, k>=0 integer, and there is a path from u and v in G whose length is < k}

Example: decision problem /corresponding language PATH

Algorithms and Decision Problems

• An algorithm accepts x if A(x)=1• An algorithm rejects x if A(x) =0Might do neither,

Language L is accepted by A: L={x, A(x)=1}

Language L is decided by A: if A accepts L and A halts on all inputs

Class PP = { L in {0,1}*: there exists algorithm A s.t. L is accepted by A in polynomial time}P - class of languages accepted by a polynomial time algorithms

Examples: shortest path, MST, sorting.

Q: Clique ? TSP? A: No one knows, big open problem

Class NP NP: class of languages accepted by a non-deterministic

algorithm in polynomial time

What does this mean? Interpretation 1: A non-deterministic algorithm has manypossible outcomes (like a probabilistic algorithm) but we require

– if x is a YES instance, then Pr(A(x)=YES) >0– if x is a NO instance, then Pr(A(x)=NO) = 1.

NP: Efficient Verification (version 2 informally )

NP : class of languages(problems) for which the YES instances have short proofs that the answer is YES.proof of polynomial size (in input size) that can be verified in polynomial time

Example: CLIQUE in NP, since for all <G,k> in CLIQUEthere exists a short proof: subset of vertices V’ of size >=k,

size of proof = O(|V|)verifying Procedure: check that for all u,v in V’, (u,v) in E

time = O(|V|2)Short Proof = Certificate

NP: Efficient Verification (formally)

NP = {L s.t. there exists polynomial -time algorithm A, and integer k>0 s.t.x in L there exists y, |y|<|x|k, A(x,y) =YES

x out L for all y, A(x,y) =NO}y= “short proof”= certificate

Language verified by A, is L= {x, there exists y s.t.A(x,y)= YES}

NP: class of polynomial time verifiable problems (EVEN IF they are hard to solve)

Theorem : P NP• Let L be in P, then there a polynomial time algorithm

which decides L, and can be converted into a verification algorithm

• HOW?BIG QUESTION: is P= NP

• Open for 25 years• What we DO know: If we can solve CLIQUE in poly-

time, then P=NP.• NP-completeness (next time)

UI

Summarize

• Decision Problems, Languages• P: polynomial time solvable decision

problems ( languages)• Certificate:evidence that allows us to verify

in polynomial time that an input is in a given language

• NP: polynomial time verifiable languages