Lecture 14: Linear Programming Relaxation andRounding

Rafael Oliveira

University of WaterlooCheriton School of Computer Science

rafael.oliveira.teaching@gmail.com

June 24, 2021

1 / 100

Overview

Part IWhy Relax & Round?

Vertex Cover

Set Cover

Conclusion

Acknowledgements

2 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?

1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

3 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?

1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

4 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!

2 Sometimes we even do that for problems in P (but we want muchmuch faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

5 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

6 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

7 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

8 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

9 / 100

Motivation - NP-hard problems

Many important optimization problems are NP-hard to solve.

What do we do when we see one?1 Find approximate solutions in polynomial time!2 Sometimes we even do that for problems in P (but we want much

much faster solutions)

Integer Linear Program (ILP):

minimize cT x

subject to Ax ≤ b

x ∈ Nn

Advantage of ILPs: very expressive language to formulateoptimization problems (capture many combinatorial optimizationproblems)

Disadvantage of ILPs: capture even NP-hard problems (thus NP-hard)

But we know how to solve LPs. Can we get partial credit in life?

10 / 100

Example

Maximum Independent Set:

G (V ,E ) graph.

Independent set S ⊆ V such that u, v ∈ S ⇒ {u, v} 6∈ E .

Integer Linear Program:

maximize∑v∈V

xv

subject to xu + xv ≤ 1 for {u, v} ∈ E

xv ∈ {0, 1} for v ∈ V

11 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

12 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

13 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

14 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

15 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

16 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.1 If solution to LP has integral values, then it is a solution to ILP and we

are done

2 If solution has fractional values, then we have to devise roundingprocedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

17 / 100

Relax... & Round!In our quest to get efficient (exact or approximate) algorithms forproblems of interest, the following strategy is very useful:

1 Formulate combinatorial optimization problem as ILP

2 Derive LP from the ILP by removing the integral constraints

This is called an LP relaxation.

3 We are still minimizing the same objective function, but over a(potentially) larger set of solutions.

opt(LP) ≤ opt(ILP)

4 Solve LP optimally using efficient algorithm.1 If solution to LP has integral values, then it is a solution to ILP and we

are done2 If solution has fractional values, then we have to devise rounding

procedure that transforms

fractional solutions → integral solutions

opt(LP) ≤ c · opt(ILP)

18 / 100

Not all LPs created equalWhen solving LP

minimize cT x

subject to Ax = b

x ≥ 0

it is important to understand geometry of feasible set & how nice thecorner points are, as they are the candidates to optimum solution.

Let P := {x ∈ Rn≥0 | Ax = b}

Vertex Solutions: a solution x ∈ P is a vertex solution if 6 ∃y 6= 0such that x + y ∈ P and x − y ∈ P

Extreme Point Solutions: x ∈ P is an extreme point solution if∃u ∈ Rn such that x is the unique optimum solution to the LP withconstraint P and objective uT x .

Basic Solutions: let supp(x) := {i ∈ [n] | xi > 0} be the set ofnonzero coordinates of x . Then x ∈ P is a basic solution ⇔ thecolumns of A indexed by supp(x) are linearly independent.

19 / 100

Not all LPs created equalWhen solving LP

minimize cT x

subject to Ax = b

x ≥ 0

it is important to understand geometry of feasible set & how nice thecorner points are, as they are the candidates to optimum solution.

Let P := {x ∈ Rn≥0 | Ax = b}

Vertex Solutions: a solution x ∈ P is a vertex solution if 6 ∃y 6= 0such that x + y ∈ P and x − y ∈ P

Extreme Point Solutions: x ∈ P is an extreme point solution if∃u ∈ Rn such that x is the unique optimum solution to the LP withconstraint P and objective uT x .

Basic Solutions: let supp(x) := {i ∈ [n] | xi > 0} be the set ofnonzero coordinates of x . Then x ∈ P is a basic solution ⇔ thecolumns of A indexed by supp(x) are linearly independent.

20 / 100

Not all LPs created equalWhen solving LP

minimize cT x

subject to Ax = b

x ≥ 0

it is important to understand geometry of feasible set & how nice thecorner points are, as they are the candidates to optimum solution.

Let P := {x ∈ Rn≥0 | Ax = b}

Vertex Solutions: a solution x ∈ P is a vertex solution if 6 ∃y 6= 0such that x + y ∈ P and x − y ∈ P

Extreme Point Solutions: x ∈ P is an extreme point solution if∃u ∈ Rn such that x is the unique optimum solution to the LP withconstraint P and objective uT x .

Basic Solutions: let supp(x) := {i ∈ [n] | xi > 0} be the set ofnonzero coordinates of x . Then x ∈ P is a basic solution ⇔ thecolumns of A indexed by supp(x) are linearly independent.

21 / 100

Not all LPs created equalWhen solving LP

minimize cT x

subject to Ax = b

x ≥ 0

it is important to understand geometry of feasible set & how nice thecorner points are, as they are the candidates to optimum solution.

Let P := {x ∈ Rn≥0 | Ax = b}

Vertex Solutions: a solution x ∈ P is a vertex solution if 6 ∃y 6= 0such that x + y ∈ P and x − y ∈ P

Extreme Point Solutions: x ∈ P is an extreme point solution if∃u ∈ Rn such that x is the unique optimum solution to the LP withconstraint P and objective uT x .

Basic Solutions: let supp(x) := {i ∈ [n] | xi > 0} be the set ofnonzero coordinates of x . Then x ∈ P is a basic solution ⇔ thecolumns of A indexed by supp(x) are linearly independent.

22 / 100

Not all LPs created equalWhen solving LP

minimize cT x

subject to Ax = b

x ≥ 0

it is important to understand geometry of feasible set & how nice thecorner points are, as they are the candidates to optimum solution.

Let P := {x ∈ Rn≥0 | Ax = b}

Vertex Solutions: a solution x ∈ P is a vertex solution if 6 ∃y 6= 0such that x + y ∈ P and x − y ∈ P

Extreme Point Solutions: x ∈ P is an extreme point solution if∃u ∈ Rn such that x is the unique optimum solution to the LP withconstraint P and objective uT x .

Basic Solutions: let supp(x) := {i ∈ [n] | xi > 0} be the set ofnonzero coordinates of x . Then x ∈ P is a basic solution ⇔ thecolumns of A indexed by supp(x) are linearly independent.

23 / 100

Part IWhy Relax & Round?

Vertex Cover

Set Cover

Conclusion

Acknowledgements

24 / 100

Vertex Cover

Setup:

Input: a graph G (V ,E ).

Output: Minimum number of vertices that “touches” all edges ofgraph. That is, minimum set S such that for each edge {u, v} ∈ Ewe have

|S ∩ {u, v}| ≥ 1.

Weighted version: associate to each vertex v ∈ V a cost cv ∈ R≥0.

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

25 / 100

Vertex Cover

Setup:

Input: a graph G (V ,E ).

Output: Minimum number of vertices that “touches” all edges ofgraph. That is, minimum set S such that for each edge {u, v} ∈ Ewe have

|S ∩ {u, v}| ≥ 1.

Weighted version: associate to each vertex v ∈ V a cost cv ∈ R≥0.

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

26 / 100

Vertex Cover

Setup:

Input: a graph G (V ,E ).

Output: Minimum number of vertices that “touches” all edges ofgraph. That is, minimum set S such that for each edge {u, v} ∈ Ewe have

|S ∩ {u, v}| ≥ 1.

Weighted version: associate to each vertex v ∈ V a cost cv ∈ R≥0.

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

27 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅

2 For each {u, v} ∈ E :1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}

3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

28 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}

3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

29 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

30 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

31 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

32 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

33 / 100

Simple 2-approximation (unweighted)

1 List edges of E in any order. Set S = ∅2 For each {u, v} ∈ E :

1 If S ∩ {u, v} = ∅, then S ← S ∪ {u, v}3 return S

Proof of correctness:

By construction, S is a vertex cover.

If added elements to S k times, then |S | = 2k and G has a matchingof size k, which means that optimum vertex cover is at least k .

Thus, we get a 2-approximation.

34 / 100

What can go wrong in the weighted case?

Original Algo Heuristic: pick lowest weight only

35 / 100

Vertex Cover - LP relaxation

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

2 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

3 Solve LP. Get optimal solution z for LP, where z = (zu)u∈V .

4 Round LP as follows: round zv to nearest integer.

36 / 100

Vertex Cover - LP relaxation

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

2 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

3 Solve LP. Get optimal solution z for LP, where z = (zu)u∈V .

4 Round LP as follows: round zv to nearest integer.

37 / 100

Vertex Cover - LP relaxation

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

2 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

3 Solve LP. Get optimal solution z for LP, where z = (zu)u∈V .

4 Round LP as follows: round zv to nearest integer.

38 / 100

Vertex Cover - LP relaxation

1 Setup ILP:

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

xu ∈ {0, 1} for u ∈ V

2 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

3 Solve LP. Get optimal solution z for LP, where z = (zu)u∈V .

4 Round LP as follows: round zv to nearest integer.

39 / 100

Vertex Cover - Analysis1 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

2 Solve LP. Get optimal solution z for LP.

3 Round zv to nearest integer. That is yv =

{1, if zv ≥ 1/2

0, if 0 ≤ zv < 1/24 y is an integral cover by construction5 each edge is covered, since given {u, v} ∈ E , at least one of zu, zv is≥ 1/2 (by feasibility of LP)

6 Cost of y is:∑u∈V

cu · yu ≤∑u∈V

cu · (2 · zu) ≤ 2 · OPT (ILP)

40 / 100

Vertex Cover - Analysis1 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

2 Solve LP. Get optimal solution z for LP.

3 Round zv to nearest integer. That is yv =

{1, if zv ≥ 1/2

0, if 0 ≤ zv < 1/2

4 y is an integral cover by construction5 each edge is covered, since given {u, v} ∈ E , at least one of zu, zv is≥ 1/2 (by feasibility of LP)

6 Cost of y is:∑u∈V

cu · yu ≤∑u∈V

cu · (2 · zu) ≤ 2 · OPT (ILP)

41 / 100

Vertex Cover - Analysis1 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

2 Solve LP. Get optimal solution z for LP.

3 Round zv to nearest integer. That is yv =

{1, if zv ≥ 1/2

0, if 0 ≤ zv < 1/24 y is an integral cover by construction

5 each edge is covered, since given {u, v} ∈ E , at least one of zu, zv is≥ 1/2 (by feasibility of LP)

6 Cost of y is:∑u∈V

cu · yu ≤∑u∈V

cu · (2 · zu) ≤ 2 · OPT (ILP)

42 / 100

Vertex Cover - Analysis1 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

2 Solve LP. Get optimal solution z for LP.

3 Round zv to nearest integer. That is yv =

{1, if zv ≥ 1/2

0, if 0 ≤ zv < 1/24 y is an integral cover by construction5 each edge is covered, since given {u, v} ∈ E , at least one of zu, zv is≥ 1/2 (by feasibility of LP)

6 Cost of y is:∑u∈V

cu · yu ≤∑u∈V

cu · (2 · zu) ≤ 2 · OPT (ILP)

43 / 100

Vertex Cover - Analysis

1 Drop integrality constraints

minimize∑u∈V

cu · xu

subject to xu + xv ≥ 1 for {u, v} ∈ E

0 ≤ xu ≤ 1 for u ∈ V

2 Solve LP. Get optimal solution z for LP.

3 Round zv to nearest integer. That is yv =

{1, if zv ≥ 1/2

0, if 0 ≤ zv < 1/24 y is an integral cover by construction5 each edge is covered, since given {u, v} ∈ E , at least one of zu, zv is≥ 1/2 (by feasibility of LP)

6 Cost of y is:∑u∈V

cu · yu ≤∑u∈V

cu · (2 · zu) ≤ 2 · OPT (ILP)

44 / 100

Part IWhy Relax & Round?

Vertex Cover

Set Cover

Conclusion

Acknowledgements

45 / 100

Set CoverSetup:

Input: a finite set U and a collection S1,S2, . . . ,Sn of subsets of U.

Output: The fewest collection of sets I ⊆ [n] such that⋃i∈I

Sj = U.

Weighted version: associate to each set Si a weight wi ∈ R≥0.

1 Setup ILP:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

xi ∈ {0, 1} for i ∈ [n]

46 / 100

Set CoverSetup:

Input: a finite set U and a collection S1,S2, . . . ,Sn of subsets of U.

Output: The fewest collection of sets I ⊆ [n] such that⋃i∈I

Sj = U.

Weighted version: associate to each set Si a weight wi ∈ R≥0.

1 Setup ILP:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

xi ∈ {0, 1} for i ∈ [n]

47 / 100

Set CoverSetup:

Input: a finite set U and a collection S1,S2, . . . ,Sn of subsets of U.

Output: The fewest collection of sets I ⊆ [n] such that⋃i∈I

Sj = U.

Weighted version: associate to each set Si a weight wi ∈ R≥0.

1 Setup ILP:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

xi ∈ {0, 1} for i ∈ [n]

48 / 100

Set Cover - Relax...

1 Obtain LP relaxation:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

0 ≤ xi ≤ 1 for i ∈ [n]

2 Suppose we end up with fractional solution z ∈ [0, 1]n when we solvethe LP above. Now need to come up with a rounding scheme.

3 Can we just round each coordinate zi to the nearest integer (like invertex cover)?

4 Not really. Say v ∈ U is in 20 sets, and we got zi = 1/20 for each ofthe sets v ∈ Si . Then rounding procedure above would not select anysuch set!

49 / 100

Set Cover - Relax...

1 Obtain LP relaxation:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

0 ≤ xi ≤ 1 for i ∈ [n]

2 Suppose we end up with fractional solution z ∈ [0, 1]n when we solvethe LP above. Now need to come up with a rounding scheme.

3 Can we just round each coordinate zi to the nearest integer (like invertex cover)?

4 Not really. Say v ∈ U is in 20 sets, and we got zi = 1/20 for each ofthe sets v ∈ Si . Then rounding procedure above would not select anysuch set!

50 / 100

Set Cover - Relax...

1 Obtain LP relaxation:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

0 ≤ xi ≤ 1 for i ∈ [n]

2 Suppose we end up with fractional solution z ∈ [0, 1]n when we solvethe LP above. Now need to come up with a rounding scheme.

3 Can we just round each coordinate zi to the nearest integer (like invertex cover)?

4 Not really. Say v ∈ U is in 20 sets, and we got zi = 1/20 for each ofthe sets v ∈ Si . Then rounding procedure above would not select anysuch set!

51 / 100

Set Cover - Relax...

1 Obtain LP relaxation:

minimize∑i∈[n]

wi · xi

subject to∑i :v∈Si

xi ≥ 1 for v ∈ U

0 ≤ xi ≤ 1 for i ∈ [n]

2 Suppose we end up with fractional solution z ∈ [0, 1]n when we solvethe LP above. Now need to come up with a rounding scheme.

3 Can we just round each coordinate zi to the nearest integer (like invertex cover)?

4 Not really. Say v ∈ U is in 20 sets, and we got zi = 1/20 for each ofthe sets v ∈ Si . Then rounding procedure above would not select anysuch set!

52 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

53 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

54 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

55 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

56 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅

4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

57 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

58 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

59 / 100

Set Cover - Rounding

1 Think of zi as the “probability” that we would pick set Si .

2 Solution z describes an “optimal probability distribution” over waysto chose the sets Si .

3 Okay, but how do we cover?

Algorithm (Random Pick)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n such that z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 for i = 1, . . . n

with probability zi , set I = I ∪ {i}

5 return I

4 Expected cost of the sets is∑n

i=1 wi · zi , which is the optimum forthe LP. But will this process cover U?

60 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

61 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

62 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

63 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

64 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

65 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

66 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

67 / 100

Analyzing Random Pick

Let’s consider the Random Pick process from point of view of v ∈ U.

v ∈ S1, . . . ,Sk (for simplicity)

As long as we select one of Si ’s above we are good (w.r.t. v)

We select Si with probability zi such that

k∑i=1

zi ≥ 1

Because z is a solution to our LP

What is probability that v is covered in Random Pick?

Definitely not 1. Think about case k = 2 and z1 = z2 = 1/2.

If had many elements like that, would expect many elementsuncovered. How to deal with this?

By perseverance! :)

68 / 100

Probability that Element is Covered

Lemma (Probability of Covering an Element)

In a sequence of k independent experiments, in which the i th experimenthas success probability pi , and

k∑i=1

pi ≥ 1

then there is a probability ≥ 1− 1/e that at least one experiment issuccessful.

Probability that no experiment is successful:

(1− p1) · (1− p2) · · · (1− pk)

1− x ≤ e−x for x ∈ [0, 1]Thus probability of failure is

k∏i=1

(1− pi ) ≤k∏

i=1

e−pi = e−p1−···−pk ≤ 1/e

69 / 100

Probability that Element is Covered

Lemma (Probability of Covering an Element)

In a sequence of k independent experiments, in which the i th experimenthas success probability pi , and

k∑i=1

pi ≥ 1

then there is a probability ≥ 1− 1/e that at least one experiment issuccessful.

Probability that no experiment is successful:

(1− p1) · (1− p2) · · · (1− pk)

1− x ≤ e−x for x ∈ [0, 1]Thus probability of failure is

k∏i=1

(1− pi ) ≤k∏

i=1

e−pi = e−p1−···−pk ≤ 1/e

70 / 100

Probability that Element is Covered

Lemma (Probability of Covering an Element)

In a sequence of k independent experiments, in which the i th experimenthas success probability pi , and

k∑i=1

pi ≥ 1

then there is a probability ≥ 1− 1/e that at least one experiment issuccessful.

Probability that no experiment is successful:

(1− p1) · (1− p2) · · · (1− pk)

1− x ≤ e−x for x ∈ [0, 1]

Thus probability of failure is

k∏i=1

(1− pi ) ≤k∏

i=1

e−pi = e−p1−···−pk ≤ 1/e

71 / 100

Probability that Element is Covered

Lemma (Probability of Covering an Element)

In a sequence of k independent experiments, in which the i th experimenthas success probability pi , and

k∑i=1

pi ≥ 1

then there is a probability ≥ 1− 1/e that at least one experiment issuccessful.

Probability that no experiment is successful:

(1− p1) · (1− p2) · · · (1− pk)

1− x ≤ e−x for x ∈ [0, 1]Thus probability of failure is

k∏i=1

(1− pi ) ≤k∏

i=1

e−pi = e−p1−···−pk ≤ 1/e72 / 100

Randomized Rounding

Algorithm (Randomized Rounding)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n s.t. z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 While there is element v ∈ U uncovered:

For i = 1, . . . , n:

with probability zi , set I = I ∪ {i}5 return I

To analyze this, need to show that we don’t execute the for loop too manytimes.

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

73 / 100

Randomized Rounding

Algorithm (Randomized Rounding)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n s.t. z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 While there is element v ∈ U uncovered:

For i = 1, . . . , n:

with probability zi , set I = I ∪ {i}5 return I

To analyze this, need to show that we don’t execute the for loop too manytimes.

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

74 / 100

Randomized Rounding

Algorithm (Randomized Rounding)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n s.t. z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 While there is element v ∈ U uncovered:

For i = 1, . . . , n:

with probability zi , set I = I ∪ {i}5 return I

To analyze this, need to show that we don’t execute the for loop too manytimes.

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

75 / 100

Randomized Rounding

Algorithm (Randomized Rounding)

1 Input: values z = (z1, . . . , zn) ∈ [0, 1]n s.t. z is a solution to our LP

2 Output: a set cover for U

3 Set I = ∅4 While there is element v ∈ U uncovered:

For i = 1, . . . , n:

with probability zi , set I = I ∪ {i}5 return I

To analyze this, need to show that we don’t execute the for loop too manytimes.

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

76 / 100

Proof of Lemma

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

Probability that for loop is executed more than ln(|U|) + t times isthe probability that there is an uncovered element after theln(|U|) + t iteration.

Let v ∈ U. For each iteration of the loop, there is a probability of 1/ethat v is not covered. (by our previous lemma)

Probability that v not covered after ln(|U|) + t iterations is(1

e

)ln(|U|)+t

=1

|U|· e−t

Union bound.

77 / 100

Proof of Lemma

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

Probability that for loop is executed more than ln(|U|) + t times isthe probability that there is an uncovered element after theln(|U|) + t iteration.

Let v ∈ U. For each iteration of the loop, there is a probability of 1/ethat v is not covered. (by our previous lemma)

Probability that v not covered after ln(|U|) + t iterations is(1

e

)ln(|U|)+t

=1

|U|· e−t

Union bound.

78 / 100

Proof of Lemma

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

Probability that for loop is executed more than ln(|U|) + t times isthe probability that there is an uncovered element after theln(|U|) + t iteration.

Let v ∈ U. For each iteration of the loop, there is a probability of 1/ethat v is not covered. (by our previous lemma)

Probability that v not covered after ln(|U|) + t iterations is(1

e

)ln(|U|)+t

=1

|U|· e−t

Union bound.

79 / 100

Proof of Lemma

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

Probability that for loop is executed more than ln(|U|) + t times isthe probability that there is an uncovered element after theln(|U|) + t iteration.

Let v ∈ U. For each iteration of the loop, there is a probability of 1/ethat v is not covered. (by our previous lemma)

Probability that v not covered after ln(|U|) + t iterations is(1

e

)ln(|U|)+t

=1

|U|· e−t

Union bound.

80 / 100

Proof of Lemma

Lemma (Probability Decay)

Let t ∈ N. The probability that the for loop will be executed more thanln(|U|) + t times is at most e−t .

Probability that for loop is executed more than ln(|U|) + t times isthe probability that there is an uncovered element after theln(|U|) + t iteration.

Let v ∈ U. For each iteration of the loop, there is a probability of 1/ethat v is not covered. (by our previous lemma)

Probability that v not covered after ln(|U|) + t iterations is(1

e

)ln(|U|)+t

=1

|U|· e−t

Union bound.

81 / 100

Cost of Rounded SolutionNow that we know we will cover with high probability, we need to boundthe cost of the solution we came up with.

At each implementation of for loop, our expected cover weight is

k∑i=1

wi · zi

After t iterations of for loop, expected weight is

t ·k∑

i=1

wi · zi

By Markov:Pr[X ≥ 2 · E[X ]] ≤ 1/2.

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

82 / 100

Cost of Rounded SolutionNow that we know we will cover with high probability, we need to boundthe cost of the solution we came up with.

At each implementation of for loop, our expected cover weight is

k∑i=1

wi · zi

After t iterations of for loop, expected weight is

t ·k∑

i=1

wi · zi

By Markov:Pr[X ≥ 2 · E[X ]] ≤ 1/2.

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

83 / 100

Cost of Rounded SolutionNow that we know we will cover with high probability, we need to boundthe cost of the solution we came up with.

At each implementation of for loop, our expected cover weight is

k∑i=1

wi · zi

After t iterations of for loop, expected weight is

t ·k∑

i=1

wi · zi

By Markov:Pr[X ≥ 2 · E[X ]] ≤ 1/2.

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

84 / 100

Cost of Rounded SolutionNow that we know we will cover with high probability, we need to boundthe cost of the solution we came up with.

At each implementation of for loop, our expected cover weight is

k∑i=1

wi · zi

After t iterations of for loop, expected weight is

t ·k∑

i=1

wi · zi

By Markov:Pr[X ≥ 2 · E[X ]] ≤ 1/2.

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

85 / 100

Cost of Rounded SolutionNow that we know we will cover with high probability, we need to boundthe cost of the solution we came up with.

At each implementation of for loop, our expected cover weight is

k∑i=1

wi · zi

After t iterations of for loop, expected weight is

t ·k∑

i=1

wi · zi

By Markov:Pr[X ≥ 2 · E[X ]] ≤ 1/2.

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

86 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

87 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

88 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

89 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

90 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

91 / 100

Cost of Rounding

Lemma (Cost of Rounding)

Given z optimal for the LP, our randomized rounding outputs, withprobability ≥ 0.45 a feasible solution to set cover with≤ 2 · (ln(|U|) + 3) · OPT (ILP) sets

1 Let t = ln(|U|) + 3. There is a probability at most e−3 < 0.05 thatwhile loop runs for more than t steps.

2 After t steps, expected weight is

ω := t ·∑

wi · zi ≤ t · OPT (ILP)

3 Markov ⇒ probability that our solution has weight ≥ 2 · ω is ≤ 1/2

4 Union bound, with probability ≤ 0.55 either run for more than ttimes, or our solution has weight ≥ 2ω

5 Thus, with probability ≥ 0.45 we stop at t iterations and constructsolution to set cover with cost ≤ 2t · OPT (ILP)

92 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

93 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

94 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

95 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.

1 If solution to LP has integral values, then it is a solution to ILP and weare done

2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

96 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.1 If solution to LP has integral values, then it is a solution to ILP and we

are done

2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

97 / 100

Putting Everything Together

1 Formulate set cover problem as ILP

2 Derive LP from the ILP LP relaxation

3 We are still minimizing the same objective function (weight of cover),but over a (potentially) larger (fractional) set of solutions.

OPT (LP) ≤ OPT (ILP)

4 Solve LP optimally using efficient algorithm.1 If solution to LP has integral values, then it is a solution to ILP and we

are done2 If have fractional values, rounding procedure

Randomized Rounding algorithm, with probability ≥ 0.45 we get

cost(rounded solution) ≤ 2 · (ln(|U|) + 3) · OPT (ILP)

98 / 100

Conclusion

Integer Linear programming - very general, and pervasive in(combinatorial) algorithmic life

ILP NP-hard

Rounding for the rescue!

Solve LP and round the solution

Deterministic rounding when solutions are niceRandomized rounding when things a bit more complicated

99 / 100

Acknowledgement

Lecture based largely on:

Lectures 7-8 of Luca’s Optimization class

See Luca’s vertex cover notes at https://lucatrevisan.github.io/teaching/cs261-11/lecture07.pdf

See Luca’s set cover notes at https://lucatrevisan.github.io/teaching/cs261-11/lecture08.pdf

100 / 100