InstructorNeelima Gupta

ngupta@cs.du.ac.in

Table of ContentsClass NP

Class NPC

Approximation Algorithms

Class NP

Class of problems for which “yes” can be verified in polynomial time.

A Verification algorithm takes as an input, a problem instance, and a certificate and decides whether it is a yes-instance.

A(x,y) = 1; iff, y is a valid certificate.

input certificateinstance

Before we elaborate on this ….

Optimization problems and Decision problemsCliqueVertex cover

Back to verification

Clique problemGiven an undirected graph G =(V,E), a

clique is a subset V’of V such that every pair of vertices is connected by an edge in E.

In the graph to the right,vertices 1,2,5 form aclique since each hasan edge to all the others.

Green ovals represent CLIQUE for this graph

G

GG

Thanks to: Sapna Grover (23, MCS '09)

Clique 1 2 3 4 5

CliqueOptimization version: Given an undirected

graph G, find a largest size Clique in it.

Decision version: Given an undirected graph G, does there exist a clique of size at most k?

“Yes” verificationGiven a set of vertices V` in G, does V` form a

Clique in G?

Vertex Cover problemGiven an undirected graph G =(V,E), a

vertex cover is a subset V’of V such that every edge in E has at least one end point in V’.

In the graph to the right,vertices 1,2,5 form aclique since each hasan edge to all the others.

Thanks to: Sapna Grover (23, MCS '09)

Vertex Cover, 1 2 3 4 5

Vertex CoverOptimization version: Given an undirected

graph G, find a smallest size vertex cover in it.

Decision version: Given an undirected graph G, does there exist a vertex cover of size at least k?

“Yes” verificationGiven a set of vertices V` in G, does V` form a

vertex cover in G?

Thanks to: Sapna Grover (23, MCS '09)

Hamiltonian Cycle ProblemA Hamiltonian cycle of an undirected graph,

G, is a simple cycle that contains every vertex.

For instance, consider the graph G

Thanks to: Sapna Grover (23, MCS '09)

Forming an HC, 1 2 3 4 5

Thanks to: Sapna Grover (23, MCS '09)

However, for the graph G`, there does not exist any Hamiltonian cycle.

1) with vertices 1,2,3,4 – not an HC

2) with vertices 1,2,5,6,3 – again not an HC

1 2

3 4 5

6

Hamiltonian Cycle ProblemThe Hamiltonian-cycle problem: given a

graph G, find a Hamiltonian cycle in it? Ham _cycle = {<G>: G is hamiltonian}.Decision version: Given an undirected graph

G, does it contain a Hamiltonian Cycle?

“Yes” verificationGiven a sequence of vertices V` in G, does V`

form a Hamiltonian cycle in G?

Class NPDefined for decision problems.

The class of decision problems for which there is a polynomially bounded nondeterministic algorithm.

“No” verification is not easyExamples:

CliqueVertex CoverHamiltonian

Class Co- NP

Class of problems for which “No” can be verified in polynomial time.

In other words, it is the set of problems whose complement is in NP.

Shortest Path“yes” and “no” both the certificates exist and

can be verified in polynomial time. Thus SP is in the intersection.

My verification algorithm will compute the shortest path in polynomial time and without explicitly using the certificate will answer “yes” or “no” as the case may be.

Class P is in the intersectionSimilarly, for every problem in P, both the

certificates exist and can be verified in polynomial time.

The verification algorithm being the polynomial time algorithm to solve the problem itself.

Hence P is in the intersection.

Thanks to: Sapna Grover (23, MCS '09)

Relation between class P, NP and Co-NP

Co-NPP

NP ∩ Co-NP NP

Thanks to: Shivam Sharma (24, MCS '09)

Instructor: Ms. Neelima Gupta

Thanks to:Prashant Singh (21,MCS '09)

Class NPCNP-completeness does not apply directly

to optimization problems, but to decision problems.

As we have seen we can obtain a decision version of an optimization problem, which will be “easier” or at least, “no harder” than the optimization problem.

Optimization problem is the harder of the two.

Thanks to: Shivam Sharma (24, MCS '09)

Defining NPC Let Q be any problem at hand, it belongs to class NPC, if,

Q NP &Q is NP hard

Thanks to: Shivam Sharma (24, MCS '09)

NP hardQ belongs to class NP-hard 1. If P NP, and2. P p Q

In step 2. we are mapping an instance of problem P to an instance of problem Q.

IP

SQ

(Yes/No)

IQ

SP

(Yes/No)

Poly. Time

Poly. Time

IP : Instance of problem P

IQ : Instance of problem Q

SP : Solution of problem P

SQ : Solution of problem Q

Thanks to: Shivam Sharma (24, MCS '09)

Transformation Characterstics :If A(Q) is yes then A(P) is yesVice versaIt should be done in polynomial time

Thanks to: Shivam Sharma (24, MCS '09)

Diagrammatically NP HARD

NPC

Thanks to: Shivam Sharma (24, MCS '09)

• Q

If all problems R NP are reducible to T, then T is NP-Hard

T

And T NP then T is NPC.

If T p Q then Q is NP hard.

Thanks to: Shivam Sharma (24, MCS '09)

3CNF p Clique (Needs to be

improved)3CNF problem : n variables constituting k clauses

3CNF is a satisfiability problem, i.e. there exists an assignment : expression results in

T.

From Cook’s Th. :The satisfiability problem is NPC.

Hence, 3CNF is NPC.

If 3CNF reduces to Clique then clique is NP hard.

Thanks to: Shivam Sharma (24, MCS '09)

Let the expression in 3CNF be: (~x v y v z) ^ (x v ~y v ~z) ^ (x v y v

z)Expression → Graph

~x y z

x

~y

~z

x

y

z

Thanks to: Shivam Sharma (24, MCS '09)

Clique thus formed:

x

y

z

Note:- There are many other possible cliques in previous mapping. This is one of the possible cliques.

Thanks to: Shivam Sharma (24, MCS '09)

Clique p Vertex Cover

Let the instance of Clique ( Ic) be <G, k>.

Reducing it to instance of VC (Ivc) be <G’, |V|-k>

where G’ : E(G’)=Edges b/w vertex pair not present in G and |V|-k is the vertex cover.

Catch behind this choice : Because it works…!!!

Thanks to: Shivam Sharma (24, MCS '09)

Green ovals represent CLIQUE for this graph

G

Thanks to: Shivam Sharma (24, MCS '09)

GG’

Thanks to: Shivam Sharma (24, MCS '09)

G’G

Big ovals represent the VC for graph G’

Thanks to: Sonia Verma (25, MCS '09)

Instructor: Ms. Neelima Gupta

Thanks to: Sonia Verma (25, MCS '09)

INDEPENDENT SETGiven an undirected graph, A subset S of V is said to be independent if no two nodes in S are joined by an edge in G.

Thanks to: Sonia Verma (25, MCS '09)

Independent set of size 2

Thanks to: Sonia Verma (25, MCS '09)

Independent set of size 3

IS is in NPC

Assignment : reduce vertex cover to IS.

Constrained Optimization ProblemsAn objective function is optimized subject to

certain constraints

Thanks to: Sufyaan Haroon (26, MCS '09)

Instructor: Ms. Neelima Gupta

Subset Sum ProblemProblem Statement

Given a finite set S = {x1, x2, ……xn of n elements with non-negative weights wi and a target t, select a subset S’ of items such that the sum of weights of elements in S’ is at most t (constraint) and,

Optimization version: the sum of weights is maximized.

Decision Version: sum of weights is at least k, for a given k.

Note: without the constraint, problem is not interesting (why?)

as you would just pick all the elements.

Subset Sum ProblemWe now prove that Subset Sum Problem is NP-Complete.

Subset Sum is in NP. For an instance <S,t>, let S’ be the certificate. Checking whether elements of S’ sum upto t and is at

least k can be done in polynomial time.

Subset Sum ProblemSubset Sum is NP HardWe show this by proving that 3-SAT is reducible to Subset Sum in polynomial time.

Given: 3-SAT formula Ф over variables x1, x2, ……xn with clauses C1, C2……Ck

Subset Sum ProblemWithout loss of generality, we make the following 2 assumptions: No clause contains both a variable and its

negation. WHY? (Because such a clause would be trivially satisfied.)

Each variable appears in at least 1 clause. WHY?(Because otherwise, it does not matter what value is assigned to it.)

Subset Sum ProblemReduction Process - through example

Consider the 3-SAT formula : Ф = C1^ C2 ^ C3

^ C4

where C1 =(x1 v x2’ v x3’) C2 =(x1’ v x2’ v x3’) C3 =(x1’ v x2’ v x3) C4 =(x1 v x2 v x3)

A satisfying assignment is <x1=0, x2=0, x3=1>

Subset Sum ProblemConstruct S and t as follows:

For each xi, add 2 integers vi, vi’ in S. Both vi and vi’ have 1 corresponding to digit xi.

If xi appears in Cj, the Cj-digit in vi = 1

If vi’ appears in Cj, the Cj-digit in vi’ = 1All other digits are zero.

Subset Sum ProblemFor all Cj, add sj and sj’ integers in S. Both

have 0’s in all digits other than the one labeled by Cj.

sj has a 1 corresponding to Cj, and sj’ has a 2 corresponding to Cj.

These integers are slack variables, used to get clause labeled digit position to add to the target value of 4.

Target t has a 1 in each digit labeled by a variable and 4 in each clause-digit.

k = t : Thus the instance of SS requires that the sum of weights of the selected elements be exactly equal (at most and at least) to t.

Subset Sum ProblemClaim: All vi and vi’ in S are uniquevi (vi’) and vj (vj’ ) will be different in most

significant positions.vi and vi’ will be different in least significant

positions. WHY? (both cannot belong to the same clause)

Subset Sum ProblemClaim: All sj and sj’ in S are unique (for reasons similar to vi and vi’)

Observation: The greatest sum of digits inany digit position is 6. This occurs in clause-digits (vi and vi’ make a contribution of 3, sj and sj’ make a contribution of 1 and 2 respectively).Conclusion: Interpretation is in base 10, so no carries would be generated.

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3)

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

Solution to 3CNF => Solution to Subset Sum

• Do the following for i = 1……n

• If xi = 1 in the assignment, include vi in S’, otherwise include vi’. In the example,

• x1=0 => x1’=1 , v1’ is selected

• x2=0 => x2’=1 , v2’ is selected

• x3=1 => x3=1 , v3 is selected

• Include Si’s as per the shortfall.

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3) Satisfying assignment x1=0, x2=0, x3=1

v1’ , v2’ and v3 are selected.

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3) Satisfying assignment x1=0, x2=0, x3=1

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3) Satisfying assignment x1=0, x2=0, x3=1

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

Thanks to: Sufyaan Haroon (26, MCS '09)

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3) Satisfying assignment x1=0, x2=0, x3=1

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

Thanks to: Sufyaan Haroon (26, MCS '09)

(x1 v x2’ v x3’) Λ (x1’ v x2’ v x3’) Λ (x1’ v x2’ v x3) Λ (x1 v x2 v x3) Satisfying assignment x1=0, x2=0, x3=1

4444111t

2000000S4'

1000000S4

0200000S3'

0100000S3

0020000S2'

0010000S2

0002000S1'

0001000S1

0011100V3'

1100100V3

0111010V2'

1000010V2

 0110001V1'

1001001V1

C4C3C2C1X3X2X1 

Solution to SS => Solution to 3CNF

• For each clause j, maximum that can be picked from Sj’s is 3. Therefore at least for one i, at least one of vi or vi’ must have been picked in S’. Set that to 1.

• Claim: there is no conflict.• Claim: the assignment is satisfying.

• Proof: do it yourself.

Proving NP-hardness by generalization

Instructor: Ms. Neelima Gupta

The 0-1 Knapsack ProblemInput

A set S of n items with weights wi and values vi

Capacity KOutput: a subset S’ of items such that

the sum of weights of items in S’ is at most K (constraint)

and the sum of values of items in S’ is maximized

0-1 Knapsack is NP-hardIt’s a generalization of subset sum

For a particular case when vi = wi, it becomes SS.

Instructor: Ms. Neelima Gupta

Set Cover ProblemProblem Statement

Given 1. A set U of n elements2. A collection S1, S2,……., Sm of subsets of U

3. A number k

Optimization version: find the smallest collection of sets that cover the elements of U.

Decision Version: Does there exist a collection of at most k of these sets that cover the elements of U.

Set Cover Problem is NP hardIt’s a generalization of vertex cover.

How?

Do it yourself

Instructor: Ms. Neelima Gupta

(Metric) Traveling Salesman ProblemProblem Statement

Given A complete graph G with non- negative edge costs ( that satisfy triangle inequality)

optimization Version: find a minimum cost cycle visiting

every vertex exactly once.

Decision Version: Does there exist a TS tour of cost <=k

TSP is NP hardReduction from Hamiltonian Cycle.Hamiltonian Cycle is a particular case of

TSP.

How?

Include an edge of G in G’ with cost 0 and an edge not in G with cost 1 and set k to 0.

Closely related problems to SS and 0-1KSBin Packing Problem

Partition problem

Bin Packing Problem

Given a set of items S = {x1…xn} each with some weight wi, pack all the items into a collection of finite number of bins each with some capacity Bi using

Optimization version: minimum number of bins.Decision version: at most k bins, for a given k.

Partition Problem

Up Next

Approximation Algorithms for some of these problems