Polyhedral approach to solve combinatorialoptimization problems

Mourad BaıouLaboratoire LIMOS, Universite Clermont II

II Escuela de Verano en Matematicas Discretas,Instituto de Sistemas Complejos de Valparaıso

08-12 Jannuary 2007

I. Polyhedral approach

Given a finite set E = {e1, e2, . . . , en}. Let F be a family of some particularsubsets of E (feasible solutions), and let c be a cost function, a mapping

c : E −→ IR.

The problem

(P ) : minimize {c(F ) def=∑

e∈F

c(e) : F ∈ F},

is called a combinatorial optimization problem.

1

Example 1 (Traveling Salesman Problem) TSPGiven a graph G = (V, E) and c ∈ IRE, find a tour T of G such that∑

e∈T c(e) is minimized. A tour is a subset of E forming a cycle in G thatvisits every node exactly once. The nodes of G may be seen as cities and cas the distances between the pairs of cities.

Example 2 (Minimal Spanning Tree Problem) MSTGiven a graph G = (V, E) and c ∈ IRE, find a tree T of G such that∑

e∈T c(e) is minimum. A tree of G is a connected, acyclic subgraph of Gspanning all the nodes V .

Example 3 (Maximum-Weight Matching Problem) MMPGiven a graph G = (V, E) and c ∈ IRE, find a matching M of G such that∑

e∈M c(e) is maximum. A matching of G is a subset of E having no nodein common.

2

v11

v

v

v

v

2

4

5 6

2e

3e4

e3ve

e ee

5

67 8

9

21

3

1

4

5

4 2

3

In red the weight associated with each edge.In bold the matching M

e

A graph H

e

The set E = {e1, . . . , e9}.F is the sets of all matchings in H.

M = {e3, e7} is a matching of weight 7.

3

Integer Polyhedra

For every element F ∈ F , associate a 0,1 vector xF ∈ {0, 1}E, defined asfollows:

xF (e) ={

1 if e ∈ F0 otherwise,

xF is called the incidence vector of F .(P ) may be rewritten as

(P ) : minimize {cTxF : xF ∈ S},

where S is the set of incidence vectors of the elements of F .

4

Example : The MMP

v11

v

v

v

v

2

4

5 6

2e

e3e4

e3ve

e ee

5

67 8

9

21

3

1

4

5

4 2

3

e

M1 M2

The incidence vector of M1 = {e3, e7} is xM1 = (0, 0, 1, 0, 0, 0, 1, 0, 0),cTxM1 = 7.

The incidence vector of M2 = {e2, e8} is xM2 = (0, 1, 0, 0, 0, 0, 0, 1, 0),cTxM2 = 4.

5

Convex hulls: The convex hull of a finite set S, denoted by conv(S), is theset of all points that are a convex combination of points in S.

Proposition 1. Let S ⊆ IRn be a finite set and let c ∈ IRn. Then

min{cTx : x ∈ S} = min{cTx : x ∈ conv(S)}.

Proof:

Immediately we have

min{cTx : x ∈ S} ≥ min{cTx : x ∈ conv(S)},

since S ⊆ conv(S).

Let S = {x1, x2, . . . , xk} and x ∈ conv(S). By definition there exists

6

positive scalars λ1, λ2, . . . , λk with∑k

i=1 λi = 1 and x =∑k

i=1 λixi, then

cTx = λ1cTx1 + λ2c

Tx2 + . . . + λkcTxk ≥ min{cTxi : i = 1, . . . , k}.2

The proposition above says that solving the combinatorial optimizationproblem (P ) reduces to a problem over the convex hull of its solutions.

The set SConv(S)

7

Polytopes:

• P = {x ∈ IRn : Ax ≤ b} where A is an m × n matrix and b ∈ IRm, iscalled a polyhedron.

• A polytope is a bounded polyhedron.

• An inequality αTx ≤ α0 is valid for a polytope P if P ⊆ {x : αTx ≤ α0}.

• The dimension of a polytope P , dim(P ), is equal to the maximumnumber of affinely independent points in P minus one.

• Let αTx ≤ α0 be a valid inequality for the polytope P . F = {x ∈P : αTx = α0} is called a face of P . F is a facet of P if dim(F ) =dim(P )− 1. A facet is maximal face.

8

• x ∈ P is an extreme point of P if x is a face of P of dimension 0.

A valid inequalitythat is a face

extreme point

A valid inequality

A facet

9

Two useful characterizations of extreme points are:

Characterization 1. x ∈ P is an extreme point of P if and only if theredo not exist x1, x2 ∈ P , x1 6= x2, such that x = 1

2x1 + 1

2x2.

Characterization 2. x ∈ P is an extreme point of P if x is theunique solution of a subsystem of inequalities defining P when replacedby equalities.

Theorem 1. A set P is a polytope if and only if there exists a finite setS such that P is the convex hull of S.

By this theorem, the optimization over conv(S) is equivalent to theoptimization over a polytope.

10

An example

Consider the following combinatorial optimization problem (P ) formulatedas an integer linear program:

max x1 + x2

subject to

P1 =

−13x1 + 28x2 ≤ 72−5x1 + 4x2 ≤ 4

9x1 − 4x2 ≤ 246x1 − 5x2 ≤ 9

x1 ≥ 0x2 ≥ 0

x1 and x2 integers

D =

−x1 + x2 ≤ 1−x1 + 2x2 ≤ 4

x1 − x2 ≤ 1x1 ≤ 4x1 ≥ 0x2 ≥ 0

Theorem 1 says that the convex hull of the solutions of (P ) is also apolytope; in our example this polytope is denoted D.

11

P1

245

245

x*violated by

A valid inequality of D

D

x*=( ),

12

Gomory-Chvatal cutting plane

Easy to see that any integral solution of P1 satisfies x1 ≤ 4. Let us showhow this can be proved.

Let x be an intgeral solution satisfying inequalities of P1.

Multiply inequality 9x1 − 4x2 ≤ 24 by 7 we obtain

63x1 − 28x2 ≤ 168,

add this inequality to inequality −13x1 + 28x2 ≤ 72, we obtain

50x1 ≤ 240,

this imply that x1 ≤ 24050 = 4, 8 ≤ b4, 8c = 4.

13

This last inequality is called Gomory-chvatal cutting plane. We can addthis inequality to derive new valid inequalities for D. This sequence ofderivations is called a cutting-plane proof.

14

The cutting-plane algorithm

We plan to solve the following integer linear program :

max {cTx : Ax ≤ b, x ∈ {0, 1}}.Ax ≤ b may contain an “exponential” number of inequalities.

Separation problem: Given a set of linear inequalities, Ax ≤ b, and avector x∗, the separation problem is to find an inequality of the set Ax ≤ bthat is violated by x∗, or to decide that Ax∗ ≤ b.

Theorem 2. (Grotschel, Lovasz and Schrijver (81)) The linearoptimization over Ax ≤ b is polynomially solvable if and only if theseparation problem for Ax ≤ b is polynomially solvable.

The cutting plane algorithm starts by optimizing cTx over a small subset ofinequalities of Ax ≤ b, call it A′x ≤ b′.

15

The algorithm:

1. Solve max {cTx : A′x ≤ b′}. Let y be the solution.

2. Apply the separation problem for Ax ≤ b and y, to find violatedinequalities if they exist,If a violated inequality is found add it to A′x ≤ b and go to step 1.

3. If y is integer valued then, y is the optimal solution,

If the final solution y is non-integer, a fractional component yi is chosen andthen we can start a branch-and-cut phase by adding a (non-valid) equationxi = 0 or xi = 1 to the current linear system P and proceeding as above,now for the systems polytopes P ∩ {x : xi = 0} and P ∩ {x : xi = 1}.

16

The matching polytope

For S ⊆ V , let δ(S), called a cut, denote the edges having one endnode inS and the other in V \ S.

The MMP can be formulated by the following linear integer program.

maximize cTx (1)∑

e∈δ(v)

x(e) ≤ 1 for all v ∈ V, (2)

x(e) ≥ 0 for all e ∈ E, (3)

x(e) ∈ {0, 1} for all e ∈ E. (4)

Let P (G) denote the polytope defined by inequalities (2) and (3).

17

MP (G) denote the convex hull of the solutions satisfying (2), (3) and (4).

The goal is to define MP (G) by a system of linear inequalities.

Is MP (G) = P (G)? The answer is NO as shown below :

12

12

12

The black edges hve value 0

Theorem 3. If G is a bipartite graph then, PM(G) = P (G).Proof: Exercise.

18

Edmonds (65) introduced the followin valid inequalities called the blossominequalities:

∑

e∈γ(S)

x(e) ≤ (|S| − 1)2

for all S ⊆ V, |S| ≥ 3 and odd, (5)

γ(S) = {e = uv ∈ E : u and v are in S}.Theorem 4. Inequalities 5 are valid for MP (G).Proof:

Take any odd set S ⊆ V . Let M a matching of G and M ′ = M ∩ γ(S).Since S has an odd cardinality, then there must exist at least one nodenot matched by the edges of M ′ and so the maximum cardinality of M ′ is(|S|−1)

2 ; that is, any incidence vector xM of a matching M in G verifies (5).2

19

Theorem 5. (The matching polytope theorem)

MP (G) =

∑e∈δ(v) x(e) ≤ 1 for all v ∈ V, (2)∑e∈γ(S) x(e) ≤ (|S|−1)

2 for all S ⊆ V, |S| ≥ 3, (5)and S odd ,

x(e) ≥ 0 for all e ∈ E (3)

20

Proof of the matching polytope theorem

The following proof was given by Lovasz (79).

Suppose thatαTx ≤ t, (6)

induces a facet of MP (G). We will show that (6) is one of the inequalities(2), (3) or (5).

Let M∗ be the set of matching in G that satisfies (6) with equality.

Remember that a facet is a maximal face, that is if every matching in M∗

satisfies with equality a valid inequality αTx ≤ t then this valid inequalityinduces the same facet as (6).

Lemma 1. αe ≥ 0 for all e ∈ E.

Proof: Suppose that α(e) < 0 for some e ∈ E. Then evey mathing in M∗

21

must satisfies x(e) = 0. Thus inequality x(e) ≥ 0 induces the same facetas (6). 2

Lemma 2. For every v ∈ V there must exist a matching in M∗ that doesnot cover v.

Proof: If for some v ∈ V every matching in M∗ covers v then, then eachmatching in M∗ satisfies

∑e∈δ(v) x(e) = 1, so

∑e∈δ(v) x(e) ≤ 1 induces

the same facet as (6). 2

Let G′ be the graph induced by the edges having α(e) > 0. Let S denoteits node set.

Lemma 3. The graph G′ is connected.

Proof: Exercise.

22

Lemma 4. Every matching in M∗ satisfies

∑

e∈γ(S)

x(e) =(|S| − 1)

2. (7)

Proof: Suppose M∗ contains matchings that do not satisfies (6). One ofthese matchings does not cover at least two nodes u and v in S. Amongthese matchings choose the one such that the distance between u and v isminimum (the number of edges in a path between u and v in G′, this pathexists from Lemma 7). Call M1 this matching.

By Lemma 1, the nodes u and v are not adjacent in G′. Thus, there mustexist a node z in the minimum path between u and v.

23

��

��

���� z

uv

S

From Lemma 2, let M2 a matching in M∗ that does not cover z. Remarkthat M2 cover both node u and v. Otherwise, M2 does not cover z and sayv, and the distance between z and v is smaller then the distance betweenu and v. By the same argument M1 must cover z.

In the graph G∗ formed by M1 ∪M2 there is a path P containing u or vthat does not contain z, since in G∗ the nodes u, v and z have degree 1.Suppose that P contains u. Define

24

M1 = (M1 \ (M1 ∩ E(P ))) ∪ (M2 ∩ E(P )),

M2 = (M2 \ (M2 ∩ E(P ))) ∪ (M1 ∩ E(P )).

By definition M1 ∪ M2 = M1 ∪M2. Thus M1 and M2 are in M∗. Theymust both satisfy (6) with equation, otherwise at least one of M1 or M2

will not satisfy it with equation, which impossible since M1 and M2 areboth in M∗.

25

M 1M 2

��

z

��

v

��

z������

v

�� � �� �� �� ��

�� ���� �� � !" #$

%�%& '�'(�()* +�+,�, -. /0 12

P

M 1

Pu

u

Pu

26

Finding facets

• To prove that a valid inequality, αx ≤ α0, is a facet for P it sufficesto find a set of dim(P ) affinely independent incidence vectors x, eachsatisfying αx ≤ α0 as an equality. This is a direct application of thedefinition of a facet.

Theorem 6. An inequality∑

e∈δ(v) x(e) ≤ 1 is a facet of MP (G) if andonly if v either has at least three distinct neighbors in G or exactly twoneighbors, u and w, and uw is not an edge of G.

Proof: Necessity. Suppose v has exactly two neighbors, u and w, andthat uw is an edge of G. Then every incidence vector that satisfies∑

e∈δ(v) x(e) ≤ 1 with equality, also satisfies with equality the blossom

27

inequality associated with S = {u, v, w}. But the reverse is not true. Thiscontradicts the fact that a facet is a maximal face.

Sufficiency. For each edge e ∈ δ(v) let Me = {e}, and for each edgee ∈ E \ δ(v) let Me = {e, f} , where f is an edge in δ(v) not incident to e.Me is a matching in G for all e ∈ E. The set of incidence vectors of Me,e ∈ E, are affinely-independent, so by definition

∑e∈δ(v) x(e) ≤ 1 induces

a facet of MP (G). 2

Exercises.

• Find the dimension of MP (G).

• Prove that all inequalities x(e) ≥ 0, define facets for MP (G).

28

Separating the blossom inequalities

The Max-Flow Min-cut Theorem

Given a directed graph G = (V, E), where eah arc e is associated with acapacity cap(e) ≥ 0, an st-flow of value v from s to t, (s, t nodes in V ) isa function x over the arcs E, such that

0 ≤ x(i, j) ≤ cap(i, j) for (i, j) ∈ E, (8)

∑

i:(i,k)∈A

x(i, k)−∑

j:(k,j)∈A

x(k, j) =

−v if k = s0 if k ∈ V, k 6= s, tv if k = t

(9)

29

s t

i1 i

ii

2

3 4

6

3

8

3

6

4

0 1

5

s t

i1 i

ii

2

3 4

9

5

2 1

6

3

8

3

7

76

Capacities A fow of value 11

30

An st-cut δ(S), is a cut separating s and t ( s ∈ S and t ∈ V \ S).

The capacity of an st-cut δ(S) is

cap(δ(S)) =∑

e=(i,j)∈E, i∈S, j∈V \Scap(e)

i3

i1 i2

i4

δ( S ) i1 i2, )( , )( i i3 4={ , }i1 i3{s, , }S=

s t

9

5

2 1

6

3

8

3

7

6its capacity is 14

31

Theorem 7. The value v of a maximum st-flow is equal to the minimumcapacity of an st-cut δ(S).

Proof: Easy to see that v ≤ cap(δ(S).The algorithm of Edmonds and karp (72) (a variant of Ford and Fulkerson(62)) finds an st-flow with value v and an st-cut with capacity v in O(n3).2

i3 4i

i2i1i1

i3

i2

4i

s t

6

8

67

0

5

22

6

1

A Max−flow of value 12

6

3

8

3

6

1s t

9

5

2

7

A cut of minimum capacity 12

32

The minimum cut problem in an undirected graph :Given an undirected graph G = (V, E), and a subset T ⊂ V ,where each edge e is associatedwith a positive weight w(e).Find a cut δ(S) in G, such that∑

e∈δ(S) w(e) is minimum.

The algorithm

• Construct a directed graph G′ from G by replacing each edge e = uv bytwo arcs (u, v) and (v, u) and let cap(u, v) = cap(v, u) = w(e).

• Fix a node r ∈ V .

• Find an rs-cut of minimum capacity for all s ∈ V \ r.

• Take the cut having the minimum capacity.

33

The minimum odd cut problem :Given an undirected graph G = (V, E), where each edge e is associatedwith a positive weight w(e). Find a cut δ(S) in G, with |S| odd, such that∑

e∈δ(S) w(e) is minimum among all odd subsets S ⊂ V .

Let δ(S) be the cut of minimum capacity in G. If |S| is odd then, we havesolved the minimum odd cut problem. Otherwise, we need the followingtheorem :

Theorem 8. Among the minimum odd cuts of G, there exits one, δ(W ),with W ⊆ S or W ⊆ (V \ S).

This allows the problem to be reduced to two problems on smaller graphs.

34

δ(S )

δ(S ) δ(S )

S1 S2

δ(S1)

S1

S

S

s

δ(S )2

S

S

S2

s

35

The separation problem of the blossom inequalities

Given a vector x∗ that satisfies∑

e∈δ(v) x∗(e) ≤ 1 for all v ∈ V and

x∗(e) ≥ 0 for all e ∈ E. The problem is to decide

if∑

e∈γ(S) x∗(e) ≤ |S|−12 for all odd subset S, or else

to find an odd subset S∗, with∑

e∈γ(S∗) x∗(e) > |S|−12 .

Define s∗v = 1−∑e∈δ(v) x∗(e) ≥ 0 for all v ∈ V .

Lemma 5. Let S ⊂ V , |S| ≥ 3 and odd.∑

e∈γ(S∗) x∗(e) > |S|−12 if and

only if∑

e∈δ(S) x∗(e) +∑

v∈S s∗(v) < 1.

36

Proof: Sum the following equalities :

s∗v = 1−∑

e∈δ(v)

x∗(e) ≥ 0 for all v ∈ S,

we obtain

2∑

e∈γ(S)

x∗(e) +∑

e∈δ(S)

x∗(e) +∑

v∈S

s∗(v) = |S|.

Add to this equality :∑

e∈γ(S) x∗(e) ≤ |S|−12 , this yields

∑

e∈δ(S)

x∗(e) +∑

v∈S

s∗(v) ≥ 1.2

37

Let G(x∗) = (V ∗, E∗) be the graph with V ∗ = V ∪ {r} and

E∗ = {e ∈ E : x∗e > 0} ∪ {e = rv : v ∈ V and s∗v > 0}

.

Let cap be the capacity function associated with E∗ defined as follows :

for all e ∈ E∗ if e is an edge joining r to a node v ∈ V , then

cap(e) = s∗(v),

otherwisecap(e) = x∗(e)

38

Let δ(S) be a cut in G∗, where r /∈ S, then its capacity is equal to

∑

e=uv∈E,u∈S,v∈V ∗\Scap(e) =

∑

e=uv∈E,u∈S,v∈V \Scap(e) +

∑

e=ur∈E,u∈S

cap(e)

=∑

e∈δ(S)

x∗(e) +∑

v∈S

s∗(v).

Thus the problem reduce to finding a minimum odd cut δ(S∗) in G∗. Ifits capacity is greater or equal to 1 then, all the blossom inequalities aresatisfied. Otherwise,

∑

e∈γ(S∗)

x∗(e) >|S| − 1

2.

39

A cutting-plane algorithm for the MMP

A graph G = (V, E) with a weight function c associated with the edges Eis given.

(0). Initialize

P ={

xe ≥ 0 for all e ∈ E,∑e∈δ(v) x(e) ≤ 1 for all v ∈ V,

(1). Solve max{cTx : x ∈ P}. Let x∗ be the solution. If x∗ is integer (a0,1-vector) then, Stop x∗ is the incidence vector of an optimal solutionto the MMP.

(2). Define G(x∗) and cap as above. Find the minimum odd cut δ(S) in

G(x∗) with respect to the capacities cap. Add x(γ(S)) ≤ |S|−12 to P

and goto step 1.

40

Note that x∗ is fractional in step 2. So by Th matching theorem polytopethere must exit a violated “blossom” inequality thus, the minimum odd cut,δ(S), found in step 2 satisfies

∑e∈δ(S) x∗(e) +

∑v∈S s∗(v) < 1 < 1.

41