Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer...

49
Decomposition in Integer Linear Programming M.V. Galati Advanced Analytics - Operations R & D, SAS Institute, Chesterbrook, PA 19087 Ted K. Ralphs Department of Industrial and System Engineering, Lehigh University, Bethlehem, PA 18015 COR@L Technical Report 04T-019-R1

Transcript of Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer...

Page 1: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Decomposition in Integer Linear Programming

M.V. Galati

Advanced Analytics - Operations R & D, SAS Institute, Chesterbrook, PA 19087

Ted K. Ralphs

Department of Industrial and System Engineering, Lehigh University, Bethlehem, PA 18015

COR@L Technical Report 04T-019-R1

Page 2: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Decomposition in Integer Linear Programming

M.V. Galati∗1 and Ted K. Ralphs†2

1Advanced Analytics - Operations R & D, SAS Institute, Chesterbrook, PA 190872Department of Industrial and System Engineering, Lehigh University, Bethlehem, PA

18015

Original Publication: September 27, 2004

Last Revised: August 16, 2005

Abstract

Both cutting plane methods and traditional decomposition methods are procedures thatcompute a bound on the optimal value of an integer linear program (ILP) by constructingan approximation to the convex hull of feasible solutions. This approximation is obtained byintersecting the polyhedron associated with the continuous relaxation, which has an explicitrepresentation, with an implicitly defined polyhedron having a description of exponential size.In this paper, we first review these classical procedures and then introduce a new class of bound-ing methods called integrated decomposition methods, in which the bound yielded by traditionalapproaches is potentially improved by introducing a second implicitly defined polyhedron. Wealso discuss the concept of structured separation, which is related to the well-known templateparadigm for dynamically generating valid inequalities and is central to our algorithmic frame-work. Finally, we briefly introduce a software framework for implementing the methods discussedin the paper and illustrate the concepts through the presentation of applications.

1 Introduction

In this paper, we discuss the principle of decomposition as it applies to the computation of boundson the value of an optimal solution to an integer linear program (ILP). Most bounding proceduresfor ILP are based on the generation of a polyhedron that approximates P, the convex hull of feasiblesolutions. Solving an optimization problem over such a polyhedral approximation, provided it fullycontains P, produces a bound that can be used to drive a branch and bound algorithm. Theeffectiveness of the bounding procedure depends largely on how well P can be approximated. Themost straightforward approximation is the continuous approximation, consisting simply of the linearconstraints present in the original ILP formulation. The bound resulting from this approximationis frequently too weak to be effective, however. In such cases, it can be improved by dynamicallygenerating additional polyhedral information that can be used to augment the approximation.

[email protected][email protected]

2

Page 3: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Traditional dynamic procedures for augmenting the continuous approximation can be groupedroughly into two categories. Cutting plane methods improve the approximation by dynamicallygenerating half-spaces containing P, i.e., valid inequalities, to form a second polyhedron, and thenintersect this second polyhedron with the continuous approximation to yield a final approximatingpolyhedron. With this approach, the valid inequalities are generated by solution of an associatedseparation problem. Generally, the addition of each valid inequality reduces the hypervolume of theapproximating polyhedron, resulting in a potentially improved bound. Because they dynamicallygenerate part of the description of the final approximating polyhedron as the intersection of half-spaces (an outer representation), we refer to cutting plane methods as outer approximation methods.

Traditional decomposition methods, on the other hand, improve the approximation by dy-namically generating the extreme points of a polyhedron containing P, which is again intersectedwith the continuous approximation, as in the cutting plane method, to yield a final approximatingpolyhedron. In this case, each successive extreme point is generated by solution of an associated op-timization problem and at each step, the hypervolume of the approximating polyhedron is increased.Because decomposition methods dynamically generate part of the description of the approximatingpolyhedron as the convex hull of a finite set (an inner representation), we refer to these methodsas inner approximation methods.

Both inner and outer methods work roughly by alternating between a procedure for computingsolution and bound information (the master problem) and a procedure for augmenting the currentapproximation (the subproblem). The two approaches, however, differ in important ways. Outermethods require that the master problem produce “primal” solution information, which then be-comes the input to the subproblem, a separation problem. Inner methods require “dual” solutioninformation, which is then used as the input to the subproblem, an optimization problem. In thissense, the two approaches can be seen as “dual” to one another. A more important difference,however, is that the valid inequalities generated by an inner method can be valid with respect toany polyhedron containing P (see Section 5), whereas the extreme points generated by an innermethod must ostensibly be from a single polyhedron. Procedures for generating new valid inequal-ities can also take advantage of knowledge of previously generated valid inequalities to furtherimprove the approximation, whereas with inner methods, such “backward-looking” procedures donot appear to be possible. Finally, the separation procedures used in the cutting plane method canbe heuristic in nature as long as it can be proven that the resulting half-spaces do actually containP. Although heuristic methods can be employed in solving the optimization problem required ofan inner method, valid bounds are only obtained when using exact optimization. On the whole,outer methods have proven to be more flexible and powerful and this is reflected in their positionas the approach of choice for solving most ILPs.

As we will show, however, inner methods do still have an important role to play. Althoughinner and outer methods have traditionally been considered separate and distinct, it is possible,in principle, to integrate them in a straightforward way. By doing so, we obtain bounds at leastas good as those yielded by either approach alone. In such an integrated method, one alternatesbetween a master problem that produces both primal and dual information, and either one of twosubproblems, one an optimization problem and the other a separation problem. This may result insignificant synergy between the subproblems, as information generated by solving the optimizationsubproblem can be used to generate cutting planes and vice versa.

The remainder of the paper is organized as follows. In Section 2, we introduce definitions andnotation. In Section 3, we describe the principle of decomposition and its application to integer

3

Page 4: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

linear programming in a traditional setting. In Section 4, we extend the traditional framework toshow how the cutting plane method can be integrated with either the Dantzig-Wolfe method or theLagrangian method to yield improved bounds. In Section 5, we discuss solution of the separationsubproblem and introduce an extension of the well-known template paradigm, called structuredseparation, inspired by the fact that separation of structured solutions is frequently easier thanseparation of arbitrary real vectors. We also introduce a decomposition-based separation algorithmcalled decompose and cut that exploits structured separation. In Section 6, we discuss some ofthe algorithms that can be used to solve the master problem. In Section 7, we describe a softwareframework for implementing the algorithms presented in the paper. Finally, in Section 8, we presentapplications that illustrate the principles discussed herein.

2 Definitions and Notation

For ease of exposition, we consider only pure integer linear programs with bounded, nonemptyfeasible regions, although the methods presented herein can be extended to more general settings.For the remainder of the paper, we consider an ILP whose feasible set is the integer vectors containedin the polyhedron Q = {x ∈ Rn | Ax ≥ b}, where A ∈ Qm×n is the constraint matrix and b ∈ Qm

is the vector of requirements. Let F = Q ∩ Zn be the feasible set and let P be the convex hull ofF . The canonical optimization problem for P is that of determining

zIP = minx∈Zn{c>x | Ax ≥ b} = min

x∈F{c>x} = min

x∈P{c>x} (1)

for a given cost vector c ∈ Qn, where zIP = ∞ if F is empty. We refer to such an ILP bythe notation ILP (P, c). In what follows, we also consider the equivalent decision version of thisproblem, which is to determine, for a given upper bound U , whether there is a member of P withobjective function value strictly better than U . We denote by OPT (P, c, U) a subroutine for solvingthis decision problem. The subroutine is assumed to return either the empty set, or a set of one ormore (depending on the situation) members of P with objective value better than U .

A related problem is the separation problem for P, which is typically already stated as a decisionproblem. Given x ∈ Rn, the problem of separating x from P is that of deciding whether x ∈ Pand if not, determining a ∈ Rn and β ∈ R such that a>y ≥ β ∀y ∈ P but a>x < β. A pair(a, β) ∈ Rn+1 such that a>y ≥ β ∀y ∈ P is a valid inequality for P and is said to be violated byx ∈ Rn if a>x < β. We denote by SEP (P, x) a subroutine that separates an arbitrary vectorx ∈ Rn from polyhedron P, returning either the empty set or a set of one or more violated validinequalities. Note that the optimization form of the separation problem is that of finding the mostviolated inequality and is equivalent to the decision form stated here.

A closely related problem is the facet identification problem, which restricts the generated in-equalities to only those that are facet-defining for P. In [30], it was shown that the facet identifica-tion problem for P is polynomially equivalent to the optimization problem for P (in the worst casesense). However, a theme that arises in what follows is that the complexity of optimization andseparation can vary significantly if either the input or the output must have known structure. If thesolution to an optimization problem is required to be integer, the problem generally becomes muchharder to solve. On the other hand, if the input vector to a separation problem is an integral vector,then the separation problem frequently becomes much easier to solve in the worst case. From thedual point of view, if the input cost vector of an optimization problem has known structure, such asbeing integral, this may make the problem easier. Requiring the output of the separation problem

4

Page 5: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

to have known structure is known as the template paradigm and may also make the separationproblem easier, but such a requirement is essentially equivalent to enlarging P. These concepts arediscussed in more detail in Section 5.

3 The Principle of Decomposition

We now formalize some of the notions described in the introduction. Implementing a branch andbound algorithm for solving an ILP requires a procedure that will generate a lower bound as closeas possible to the optimal value zIP . The most commonly used method of bounding is to solvethe linear programming (LP) relaxation obtained by removing the integrality requirement from theILP formulation. The LP Bound is given by

zLP = minx∈Rn{c>x | Ax ≥ b} = min

x∈Q{c>x}, (2)

and is obtained by solving a linear program with the original objective function c over the poly-hedron Q. It is clear that zLP ≤ zIP since P ⊆ Q. This LP relaxation is usually much easier tosolve than the original ILP, but zLP may be arbitrarily far away from zIP in general, so we needto consider more effective procedures.

In most cases, the description of Q is small enough that it can be represented explicitly andthe bound computed using a standard linear programming algorithm. To improve the LP bound,decomposition methods construct a second approximating polyhedron that can be intersected withQ to form a better approximation. Unlike Q, this second polyhedron usually has a description ofexponential size, and we must generate portions of its description dynamically. Such a dynamicprocedure is the basis for both cutting plane methods, which generate an outer approximation, andfor traditional decomposition methods, such as the Dantzig-Wolfe method [19] and the Lagrangianmethod [22, 14], which generate inner approximations.

For the remainder of this section, we consider the relaxation of (1) defined by

minx∈Zn{c>x | A′x ≥ b′} = min

x∈F ′{c>x} = min

x∈P ′{c>x}, (3)

where F ⊂ F ′ = {x ∈ Zn | A′x ≥ b′} for some A′ ∈ Qm′×n, b′ ∈ Qm′ and P ′ is the convexhull of F ′. Along with P ′ is associated a set of side constraints [A′′, b′′] ∈ Qm′′×(n+1) such thatQ = {x ∈ Rn | A′x ≥ b′, A′′x ≥ b′′}. We denote by Q′ the polyhedron described by the inequalities[A′, b′] and by Q′′ the polyhedron described by the inequalities [A′′, b′′]. Thus, Q = Q′ ∩ Q′′ andF = {x ∈ Zn | x ∈ P ′∩Q′′}. For the decomposition to be effective, we must have that P ′∩Q′′ ⊂ Q,so that the bound obtained by optimizing over P ′ ∩ Q′′ is at least as good as the LP bound andstrictly better for some objective functions. The description of Q′′ must also be “small” so that wecan construct it explicitly. Finally, we assume that there exists an effective algorithm for optimizingover P ′ and thereby, for separating arbitrary real vectors from P ′. We are deliberately using theterm effective here to denote an algorithm that has an acceptable average-case running time, sincethis is more relevant than worst-case behavior in our computational framework.

Traditional decomposition methods can all be viewed as techniques for iteratively computingthe bound

zD = minx∈P ′{c>x | A′′x ≥ b′′} = min

x∈F ′∩Q′′{c>x} = min

x∈P ′∩Q′′{c>x}. (4)

5

Page 6: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

In Sections 3.1–3.3 below, we review the cutting plane method, the Dantzig-Wolfe method, andthe Lagrangian method, all classical approaches that can be used to compute this bound. Thecommon perspective motivates Section 4, where we consider a new class of decomposition methodscalled integrated decomposition methods, in which both inner and outer approximation techniquesare used in tandem. In both this section and the next, we describe the methods at a high level andleave until later sections the discussion of how the master problem and subproblems are solved. Toillustrate the effect of applying the decomposition principle, we now introduce two examples thatwe build on throughout the paper. The first is a simple generic ILP.

Example 1 Let the following be the formulation of a given ILP:

min x1,

7x1 − x2 ≥ 13, (5)

x2 ≥ 1, (6)

−x1 + x2 ≥ −3, (7)

−4x1 − x2 ≥ −27, (8)

−x2 ≥ −5, (9)

0.2x1 − x2 ≥ −4, (10)

−x1 − x2 ≥ −8, (11)

−0.4x1 + x2 ≥ 0.3, (12)

x1 + x2 ≥ 4.5, (13)

3x1 + x2 ≥ 9.5, (14)

0.25x1 − x2 ≥ −3, (15)

x ∈ Z2. (16)

In this example, we let

P = conv{x ∈ R2 | x satisfies (5)− (16)},Q′ = {x ∈ R2 | x satisfies (5)− (10)},Q′′ = {x ∈ R2 | x satisfies (11)− (15)}, and

P ′ = conv(Q′ ∩ Z2).

In Figure 1(a), we show the associated polyhedra, where the set of feasible solutions F = Q′ ∩Q′′ ∩ Z2 = P ′ ∩ Q′′ ∩ Z2 and P = conv(F). Figure 1(b) depicts the continuous approximationQ′ ∩Q′′, while Figure 1(c) shows the improved approximation P ′ ∩Q′′. For the objective functionin this example, optimization over P ′ ∩ Q′′ leads to an improvement over the LP bound obtainedby optimization over Q.

6

Page 7: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

(a) (b) (c)

(2,1)P

P′

Q′′

Q′

P

Q′ ∩ Q′′

P

P′ ∩ Q′′

(2,1) (2,1)

Figure 1: Polyhedra (Example 1)

In our second example, we consider the classical Traveling Salesman Problem (TSP), a well-knowncombinatorial optimization problem. The TSP is in the complexity class NP-hard, but lends itselfwell to the application of the principle of decomposition, as the standard formulation contains anexponential number of constraints and has a number of well-solved combinatorial relaxations.

Example 2 The Traveling Salesman Problem is that of finding a minimum cost tour in anundirected graph G with vertex set V = {0, 1, ..., |V | − 1} and edge set E. We assume withoutloss of generality that G is complete. A tour is a connected subgraph for which each node hasdegree 2. The TSP is then to find such a subgraph of minimum cost, where the cost is the sumof the costs of the edges comprising the subgraph. With each edge e ∈ E, we therefore associatea binary variable xe, indicating whether edge e is part of the subgraph, and a cost ce ∈ R. Letδ(S) = {{i, j} ∈ E | i ∈ S, j /∈ S}, E(S : T ) = {{i, j} | i ∈ S, j ∈ T}, E(S) = E(S : S) andx(F ) =

∑e∈F xe. Then an ILP formulation of the TSP is as follows:

min∑e∈E

cexe,

x(δ({i})) = 2 ∀i ∈ V, (17)

x(E(S)) ≤ |S| − 1 ∀S ⊂ V, 3 ≤ |S| ≤ |V | − 1, (18)

0 ≤ xe ≤ 1 ∀e ∈ E, (19)

xe ∈ Z ∀e ∈ E. (20)

The continuous approximation, referred to as the TSP polyhedron, is then

P = conv{x ∈ RE | x satisfies (17)− (20)}.

The equations (17) are the degree constraints, which ensure that each vertex has degree two in thesubgraph, while the inequalities (18) are known as the subtour elimination constraints (SECs) andenforce connectivity. Since there are an exponential number of SECs, it is impossible to explicitly

7

Page 8: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

construct the LP relaxation of TSP for large graphs. Following the pioneering work of Held andKarp [35], however, we can apply the principle of decomposition by employing the well-knownMinimum 1-Tree Problem, a combinatorial relaxation of TSP.

A 1-tree is a tree spanning V \ {0} plus two edges incident to vertex 0. A 1-tree is hence asubgraph containing exactly one cycle through vertex 0. The Minimum 1-Tree Problem is to finda 1-tree of minimum cost and can thus be formulated as follows:

min∑e∈E

cexe,

x(δ({0})) = 2, (21)

x(E(V \ {0})) = |V | − 2, (22)

x(E(S)) ≤ |S| − 1 ∀S ⊂ V \ {0}, 3 ≤ |S| ≤ |V | − 1, (23)

xe ∈ {0, 1} ∀e ∈ E. (24)

A minimum cost 1-tree can be obtained easily as the union of a minimum cost spanning treeof V \ {0} plus two cheapest edges incident to vertex 0. For this example, we thus let P ′ =conv({x ∈ RE | x satisfies (21) − (24)}) be the 1-Tree Polyhedron, while the degree and boundconstraints comprise the polyhedron Q′′ = {x ∈ RE | x satisfies (17) and (19)} and Q′ = {x ∈RE | x satisfies (18)}. Note that the bound constraints appear in the descriptions of both polyhedrafor computational convenience. The set of feasible solutions to TSP is then F = P ′ ∩Q′′ ∩ ZE .

3.1 Cutting Plane Method

Using the cutting plane method, the bound zD can be obtained by dynamically generating portionsof an outer description of P ′. Let [D, d] denote the set of facet-defining inequalities of P ′, so that

P ′ = {x ∈ Rn | Dx ≥ d}. (25)

Then the cutting plane formulation for the problem of calculating zD can be written as

zCP = minx∈Q′′

{c>x | Dx ≥ d}. (26)

This is a linear program, but since the set of valid inequalities [D, d] is potentially of exponentialsize, we dynamically generate them by solving a separation problem. An outline of the method ispresented in Figure 2.

In Step 2, the master problem is a linear program whose feasible region is the current outerapproximation PtO, defined by a set of initial valid inequalities plus those generated dynamically inStep 3. Solving the master problem in iteration t, we generate the relaxed (primal) solution xtCPand a valid lower bound. In the figure, the initial set of inequalities is taken to be those of Q′′,since it is assumed that the facet-defining inequalities for P ′, which dominate those of Q′, can begenerated dynamically. In practice, however, this initial set may be chosen to include those of Q′or some other polyhedron, on an empirical basis.

In Step 3, we solve the subproblem, which is to generate a set of improving valid inequalities,i.e., valid inequalities that improve the bound when added to the current approximation. This stepis usually accomplished by applying one of the many known techniques for separating xtCP from

8

Page 9: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Cutting Plane Method

Input: An instance ILP (P, c).Output: A lower bound zCP on the optimal solution value for the instance, andxCP ∈ Rn such that zCP = c>xCP .

1. Initialize: Construct an initial outer approximation

P0O = {x ∈ Rn | D0x ≥ d0} ⊇ P, (27)

where D0 = A′′ and d0 = b′′, and set t← 0.

2. Master Problem: Solve the linear program

ztCP = minx∈Rn{c>x | Dtx ≥ dt} (28)

to obtain the optimal value ztCP = minx∈PtO{c>x} ≤ zIP and optimal primal

solution xtCP .

3. Subproblem: Call the subroutine SEP (P, xtCP ) to generate a set of poten-tially improving valid inequalities [D, d] for P, violated by xtCP .

4. Update: If violated inequalities were found in Step 3, set [Dt+1, dt+1] ←[Dt dt

D d

]to form a new outer approximation

Pt+1O = {x ∈ Rn | Dt+1x ≤ dt+1} ⊇ P, (29)

and set t← t+ 1. Go to Step 2.

5. If no violated inequalities were found, output zCP = ztCP ≤ zIP and xCP =xtCP .

Figure 2: Outline of the cutting plane method

9

Page 10: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

P. The algorithmic details of the generation of valid inequalities are covered more thoroughly inSection 5, so the unfamiliar reader may wish to refer to this section for background or to [1] for acomplete survey of techniques. It is well known that violation of xtCP is a necessary condition foran inequality to be improving, and hence, we generally use this condition to judge the potentialeffectiveness of generated valid inequalities. However, this condition is not sufficient and unless theinequality separates the entire optimal face of PtO, it will not actually be improving. Because wewant to refer to these results later in the paper, we state them formally as theorem and corollarywithout proof. See [59] for a thorough treatment of the theory of linear programming that leads tothis result.

Theorem 1 Let F be the face of optimal solutions to an LP over a nonempty, bounded polyhedronX with objective function vector f . Then (a, β) is an improving inequality for X with respect to f ,i.e.,

min{f>x | x ∈ X, a>x ≥ β} > min{f>x | x ∈ X}, (30)

if and only if a>y < β for all y ∈ F .

Corollary 1 If (a, β) is an improving inequality for X with respect to f , then a>x < β, where xis any optimal solution to the linear program over X with objective function vector f .

Even in the case when the optimal face cannot be separated in its entirety, the augmented cuttingplane LP must have a different optimal solution, which in turn may be used to generate morepotential improving inequalities. Since the condition of Theorem 1 is difficult to verify, one typicallyterminates the bounding procedure when increases resulting from additional inequalities become“too small.”

If we start with the continuous approximation P0O = Q′′ and generate only facet-defining in-

equalities of P ′ in Step 3, then the procedure described here terminates in a finite number of stepswith the bound zCP = zD (see [52]). Since PtO ⊇ P ′ ∩ Q′′ ⊇ P, each step yields an approximationfor P, along with a valid bound. In Step 3, we are permitted to generate any valid inequality forP, however, not just those that are facet-defining for P ′. In theory, this means that the cuttingplane method can be used to compute the bound zIP exactly. However, this is rarely practical.

To illustrate the cutting plane method, we show how it could be applied to generate the boundzD for the ILPs of Examples 1 and 2. Since we are discussing the computation of the bound zD,we only generate facet-defining inequalities for P ′ in these examples. We discuss more generalscenarios later in the paper.

Example 1 (Continued) We define the initial outer approximation to be P0O = Q′ ∩Q′′ = {x ∈

R2 | x satisfies (5)− (15)}, the continuous approximation.

Iteration 0: Solving the master problem over P0O, we find an optimal primal solution x0

CP =(2.25, 2.75) with bound z0

CP = 2.25, as shown in Figure 3(a). We then call the subroutineSEP (P, x0

CP ), generating facet-defining inequalities of P ′ that are violated by x0CP . One such

facet-defining inequality, 3x1 − x2 ≥ 5, is pictured in Figure 3(a). We add this inequality to forma new outer approximation P1

O.

Iteration 1: We again solve the master problem, this time over P1O, to find an optimal primal

solution x1CP = (2.42, 2.25) and bound z1

CP = 2.42, as shown in Figure 3(b). We then call the

10

Page 11: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

(a) (b)

x0CP = (2.25, 2.75)

(2, 1) (2, 1)

P

P′

P0O = Q′ ∩ Q′′

P

P′

P1O = P0

O ∩ {x ∈ Rn | 3x1 − x2 ≥ 5}

x1CP = (2.42, 2.25)

Figure 3: Cutting plane method (Example 1)

subroutine SEP (P, x1CP ). However, as illustrated in Figure 3(b), there are no more facet-defining

inequalities violated by x1CP . In fact, further improvement in the bound would necessitate the

addition of valid inequalities violated by points in P ′. Since we are only generating facets of P ′ inthis example, the method terminates with bound zCP = 2.42 = zD.

We now consider the use of the cutting plane method for generating the bound zD for the TSP ofExample 2. Once again, we only generate facet-defining inequalities for P ′, the 1-tree polyhedron.

Example 2 (Continued) We define the initial outer approximation to be comprised of thedegree constraints and the bound constraints, so that

P0O = Q′′ = {x ∈ RE | x satisfies (17) and (19)}.

The bound zD is then obtained by optimizing over the intersection of the 1-tree polyhedron withthe polyhedron Q′′ defined by constraints (17) and (19). Note that because the 1-tree polyhedronhas integer extreme points, we have that zD = zLP in this case. To calculate zD, however, we mustdynamically generate violated facet-defining inequalities (the SECs (23)) of the 1-tree polyhedronP ′ defined earlier. Given a vector x ∈ RE satisfying (17) and (19), the problem of finding aninequality of the form (23) violated by x is equivalent to the well-known minimum cut problem,which can be nominally solved in O(|V |4) [53]. We can use this approach to implement Step 3 ofthe cutting plane method and hence compute the bound zD effectively. As an example, considerthe vector x pictured graphically in Figure 4, obtained in Step 2 of the cutting plane method. Inthe figure, only edges e for which xe > 0 are shown. Each edge e is labeled with the value xe,except for edges e with xe = 1. The circled set of vertices S = {0, 1, 2, 3, 7} define a SEC violatedby x, since x(E(S)) = 4.6 > 4.0 = |S| − 1.

11

Page 12: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

0

1

2

0.6

3

0.2

0.8

0.2

4

5

6

0.8

7

0.8

8

9

0.6

10

11

0.4

0.2

12

0.2

0.2

0.2

13

0.4

0.60.8

14

0.6

0.2

0.2

15

0.2

0.2

0.2

0.8

0.6

Figure 4: Finding violated inequalities in the cutting plane method (Example 2)

3.2 Dantzig-Wolfe Method

In the Dantzig-Wolfe method, the bound zD can be obtained by dynamically generating portionsof an inner description of P ′ and intersecting it with Q′′. Consider Minkowski’s Theorem, whichstates that every bounded polyhedron is finitely generated by its extreme points [52]. Let E ⊆ F ′be the set of extreme points of P ′, so that

P ′ = {x ∈ Rn | x =∑s∈E

sλs,∑s∈E

λs = 1, λs ≥ 0 ∀s ∈ E}. (31)

Then the Dantzig-Wolfe formulation for computing the bound zD is

zDW = minx∈Rn{c>x | A′′x ≥ b′′, x =

∑s∈E

sλs,∑s∈E

λs = 1, λs ≥ 0 ∀s ∈ E}. (32)

By substituting out the original variables, this formulation can be rewritten in the more familiarform

zDW = minλ∈RE+

{c>(∑s∈E

sλs) | A′′(∑s∈E

sλs) ≥ b′′,∑s∈E

λs = 1}. (33)

This is a linear program, but since the set of extreme points E is potentially of exponential size,we dynamically generate those that are relevant by solving an optimization problem over P ′. Anoutline of the method is presented in Figure 5.

In Step 2, we solve the master problem, which is a restricted linear program obtained bysubstituting E t for E in (33). In Section 6, we discuss several alternatives for solving this LP.In any case, solving it results in a primal solution λtDW , and a dual solution consisting of thedual multipliers utDW on the constraints corresponding to [A′′, b′′] and the multiplier αtDW on theconvexity constraint. The dual solution is needed to generate the improving columns in Step 3. Ineach iteration, we are generating an inner approximation, PtI ⊆ P ′, the convex hull of E t. ThusPtI ∩ Q′′ may or may not contain P and the bound returned from the master problem in Step 2,ztDW , provides an upper bound on zDW . Nonetheless, it is easy to show (see Section 3.3) that an

12

Page 13: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Dantzig-Wolfe Method

Input: An instance ILP (P, c).Output: A lower bound zDW on the optimal solution value for the instance, a primalsolution λDW ∈ RE , and a dual solution (uDW , αDW ) ∈ Rm′′+1.

1. Initialize: Construct an initial inner approximation

P0I = {

∑s∈E0

sλs |∑s∈E0

λs = 1, λs ≥ 0 ∀s ∈ E0, λs = 0 ∀s ∈ E \ E0} ⊆ P ′ (34)

from an initial set E0 of extreme points of P ′ and set t← 0.

2. Master Problem: Solve the Dantzig-Wolfe reformulation

ztDW = minλ∈RE+

{c>(∑s∈E

sλs) | A′′(∑s∈E

sλs) ≥ b′′,∑s∈E

λs = 1, λs = 0 ∀s ∈ E \ E t}

(35)to obtain the optimal value ztDW = minPtI∩Q′′ c

>x ≥ zDW , an optimal primal

solution λtDW ∈ RE+, and an optimal dual solution (utDW , αtDW ) ∈ Rm′′+1.

3. Subproblem: Call the subroutine OPT (P ′, c>− (utDW )>A′′, αtDW ), generat-ing a set of E of improving members of E with negative reduced cost, wherethe reduced cost of s ∈ E is

rc(s) = (c> − (utDW )>A′′)s− αtDW . (36)

If s ∈ E is the member of E with smallest reduced cost, then ztDW = rc(s) +αtDW + (utDW )>b′′ ≤ zDW provides a valid lower bound.

4. Update: If E 6= ∅, set E t+1 ← E t ∪ E to form the new inner approximation

Pt+1I = {

∑s∈Et+1

sλs |∑

s∈Et+1

λs = 1, λs ≥ 0 ∀s ∈ E t+1, λs = 0 ∀s ∈ E\E t+1} ⊆ P ′,

(37)and set t← t+ 1. Go to Step 2.

5. If E = ∅, output the bound zDW = ztDW = ztDW , λDW = λtDW , and(uDW , αDW ) = (utDW , α

tDW ).

Figure 5: Outline of the Dantzig-Wolfe method

13

Page 14: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

optimal solution to the subproblem solved in Step 3 yields a valid lower bound. In particular, if sis a member of E with the smallest reduced cost in Step 3, then

ztDW = c>s+ (utDW )>(b′′ −A′′s) (38)

is a valid lower bound. This means that, in contrast to the cutting plane method, where a validlower bound is always available, the Dantzig-Wolfe method only yields a valid lower bound whenthe subproblem is solved to optimality, i.e., the optimization version is solved, as opposed to thedecision version. This need not be done in every iteration, as described below.

In Step 3, we search for improving members of E , where, as in the previous section, this meansmembers that when added to E t yield an improved bound. It is less clear here, however, whichbound we would like to improve, ztDW or ztDW . A necessary condition for improving ztDW is thegeneration of a column with negative reduced cost. In fact, if one considers (38), it is clear thatthis condition is also necessary for improvement of ztDW . However, we point out again that thesubproblem must be solved to optimality in order to update the bound ztDW . In either case,however, we are looking for members of E with negative reduced cost. If one or more such membersexist, we add them to E t and iterate.

An area that deserves some deeper investigation is the relationship between the solution obtainedby solving the reformulation (35) and the solution that would be obtained by solving an LP directlyover PtI ∩ Q′′ with the objective function c. Consider the primal optimal solution λtDW , which werefer to as an optimal decomposition. If we combine the members of E t using λtDW to obtain anoptimal fractional solution

xtDW =∑s∈Et

s(λtDW )s, (39)

then we see that ztDW = c>xtDW . In fact, xtDW ∈ PtI ∩ Q′′ is an optimal solution to the linearprogram solved directly over PtI ∩Q′′ with objective function c.

The optimal fractional solution plays an important role in the integrated methods to be intro-duced later. To illustrate the Dantzig-Wolfe method and the role of the optimal fractional solutionin the method, we show how to apply it to generate the bound zD for the ILP of Example 1.

Example 1 (Continued) For the purposes of illustration, we begin with a randomly generatedinitial set of points E0 = {(4, 1), (5, 5)}. Taking their convex hull, we form the initial inner approx-imation P0

I = conv(E0), as illustrated in Figure 6(a).

Iteration 0. Solving the master problem with inner polyhedron P0I , we obtain an optimal pri-

mal solution (λ0DW )(4,1) = 0.75, (λ0

DW )(5,5) = 0.25, x0DW = (4.25, 2), and bound z0

DW = 4.25. Sinceconstraint (12) is binding at x0

DW , the only nonzero component of u0DW is (u0

DW )(12) = 0.28, whilethe dual variable associated with the convexity constraint has value α0

DW = 4.17. All other dualvariables have value zero. Next, we search for an extreme point of P ′ with negative reduced cost, bysolving the subproblem OPT (P ′, c>− (utDW )>A′′, α0

DW ). From Figure 6(a), we see that s = (2, 1).This gives a valid lower bound z0

DW = 2.03. We add the corresponding column to the restrictedmaster and set E1 = E0 ∪ {(2, 1)}.

Iteration 1. The next iteration is depicted in Figure 6(b). First, we solve the master problemwith inner polyhedron P1

I = conv(E1) to obtain (λ1DW )(5,5) = 0.21, (λ1

DW )(2,1) = 0.79, x1DW =

14

Page 15: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

(b) (c)(a)

x2DW = (2.42, 2.25)

P

P′ P′

c> − u>A”c>

c> − u>A”

c> − u>A”

(2, 1) (2, 1) (2, 1)

Q′′ Q′′

P

Q′′

P

P′

P1I = conv(E1) ⊂ P′ P2

I = conv(E2) ⊂ P′P0I = conv(E0) ⊂ P′

s = (3, 4)

E1

s = (2, 1)

E0

E2

x0DW = (4.25, 2) x1DW = (2.64, 1.86)

Figure 6: Dantzig-Wolfe method (Example 1)

(2.64, 1.86), and bound and z1DW = 2.64. This also provides the dual solution (u1

DW )(13) = 0.43

and α1DW = 0.71 (all other dual values are zero). Solving OPT (P ′, c> − u1

DWA′′, α1

DW ), we obtains = (3, 4), and z1

DW = 1.93. We add the corresponding column to the restricted master and setE2 = E1 ∪ {(3, 4)}.

Iteration 2 The final iteration is depicted in Figure 6(c). Solving the master problem oncemore with inner polyhedron P2

I = conv(E2), we obtain (λ2DW )(2,1) = 0.58 and (λ2

DW )(3,4) = 0.42,x2DW = (2.42, 2.25), and bound z2

DW = 2.42. This also provides the dual solution (u2DW )(14) = 0.17

and α2DW = 0.83. Solving OPT (P ′.c> − u2

DWA′′, α2

DW ), we conclude that E = ∅. We thereforeterminate with the bound zDW = 2.42 = zD.

As a further brief illustration, we return to the TSP example introduced earlier.

Example 2 (Continued) As we noted earlier, the Minimum 1-Tree Problem can be solved bycomputing a minimum cost spanning tree on vertices V \ {0}, and then adding two cheapest edgesincident to vertex 0. This can be done in O(|E| log |V |) using standard algorithms. In applying theDantzig-Wolfe method to compute zD using the decomposition described earlier, the subproblemto be solved in Step 3 is a Minimum 1-Tree Problem. Because we can solve this problem effectively,we can apply the Dantzig-Wolfe method in this case. As an example of the result of solving theDantzig-Wolfe master problem (35), Figure 7 depicts an optimal fractional solution (a) to a Dantzig-Wolfe master LP and the six extreme points 7(b-g) of the 1-tree polyhedron P ′, with nonzero weightcomprising an optimal decomposition. We return to this figure later in Section 4.

Now consider the set S(u, α), defined as

S(u, α) = {s ∈ E | (c> − u>A′′)s = α}, (40)

where u ∈ Rm′′ and α ∈ R. The set S(utDW , αtDW ) is the set of members of E with reduced cost

15

Page 16: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

0

0

0 0

0

11

15

14

13 12

0

0

1

1

1

1

1 1

1

2

22

2

2

2

2

3

3

33

33

3

4

4

4

4

4

4

4

5

5

5 5

5

5

5

6

6

6

66

6

6

7 77

7

7

77

8

8

8

8

8

10

88

9

9 9

9 9

9

9

11

1

1

1

1 0

00 0

0

0

111

11

1 1

111

1

1

1 1

11

12

1

2

222

21

11

111

33

3

3

33

1 11

1 1

44 4

41

4

4

11

111

1

5 5

5

5

5 5

0.2

0.5

0.5

0.5

0.3

0.2

0.5

0.5

0.5

0.70.3

0.3

(d) λ2 = 0.2

(e) λ3 = 0.1 (f) λ4 = 0.1 (g) λ5 = 0.1

(a) x

(b) λ0 = 0.3 (c) λ1 = 0.2

Figure 7: Dantzig-Wolfe method (Example 2)

zero at optimality for (35) in iteration t. It follows that conv(S(utDW , αtDW )) is in fact the face of

optimal solutions to the linear program solved over PtI with objective function c> − u>A′′. Thisline of reasoning culminates in the following theorem tying together the set S(utDW , α

tDW ) defined

above, the vector xtDW , and the optimal face of solutions to the LP over the polyhedron PtI ∩Q′′.

Theorem 2 conv(S(utDW , αtDW )) is a face of PtI and contains xtDW .

Proof. We first show that conv(S(utDW , αtDW )) is a face of PtI . Observe that

(c> − (utDW )>A′′, αtDW )

defines a valid inequality for PtI since αtDW is the optimal value for the problem of minimizing overPtI with objective function c> − (utDW )>A′′. Thus, the set

G = {x ∈ PtI | (c> − (utDW )>A′′)x = αtDW }, (41)

is a face of PtI that contains S(utDW , αtDW ). We will show that conv(S(utDW , α

tDW )) = G. Since

G is convex and contains S(utDW , αtDW ), it also contains conv(S(utDW , α

tDW )), so we just need to

show that conv(S(utDW , αtDW )) contains G. We do so by observing that the extreme points of G

are elements of S(utDW , αtDW ). By construction, all extreme points of PtI are members of E and

the extreme points of G are also extreme points of PtI . Therefore, the extreme points of G must

16

Page 17: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

be members of E and contained in S(utDW , αtDW ). The claim follows and conv(S(utDW , α

tDW )) is a

face of PtI .

The fact that xtDW ∈ conv(S(utDW , αtDW )) follows from the fact that xtDW is a convex combination

of members of S(utDW , αtDW ).

An important consequence of Theorem 2 is that the face of optimal solutions to the LP over thepolyhedron PtI ∩ Q′′ is actually contained in conv(S(utDW , α

tDW )) ∩ Q′′, as stated in the following

corollary.

Corollary 2 If F is the face of optimal solutions to the linear program solved directly over PtI ∩Q′′with objective function vector c, then F ⊆ conv(S(utDW , α

tDW )) ∩Q′′.

Proof. Let x ∈ F be given. Then we have that x ∈ PtI ∩Q′′ by definition, and

c>x = αtDW + (utDW )>b′′ = αtDW + (utDW )>A′′x, (42)

where the first equality in this chain is a consequence of strong duality and the last is a consequenceof complementary slackness. Hence, it follows that (c> − (utDW )>A′′)x = αtDW and the result isproven.

Hence, each iteration of the method not only produces the primal solution xtDW ∈ PtI ∩ Q′′, butalso a dual solution (utDW , α

tDW ) that defines a face conv(S(utDW , α

tDW )) of PtI that contains the

entire optimal face of solutions to the LP solved directly over PtI ∩ Q′′ with the original objectivefunction vector c.

When no column with negative reduced cost exists, the two bounds must be equal to zD andwe stop, outputting both the primal solution λDW , and the dual solution (uDW , αDW ). It followsfrom the results proven above that in the final iteration, any column of (35) with reduced costzero must in fact have a cost of αDW = zD − u>DW b′′ when evaluated with respect to the modifiedobjective function c> − u>DWA′′. In the final iteration, we can therefore strengthen the statementof Theorem 2, as follows.

Theorem 3 conv(S(uDW , αDW )) is a face of P ′ and contains xDW .

The proof follows along the same lines as Theorem 2. As before, we can also state the followingimportant corollary.

Corollary 3 If F is the face of optimal solutions to the linear program solved directly over P ′∩Q′′with objective function vector c, then F ⊆ conv(S(uDW , αDW )) ∩Q′′.

Thus, conv(S(uDW , αDW )) is actually a face of P ′ that contains xDW and the entire face of optimalsolutions to the LP solved over P ′∩Q′′ with objective function c. This fact provides strong intuitionregarding the connection between the Dantzig-Wolfe method and the cutting plane method andallows us to regard Dantzig-Wolfe decomposition as either a procedure for producing the boundzD = c>xDW from primal solution information or the bound zD = c>s + u>DW (b′′ − A′′s), wheres is any member of S(uDW , αDW ), from dual solution information. This fact is important in thenext section, as well as later when we discuss integrated methods.

The exact relationship between S(uDW , αDW ), the polyhedron P ′ ∩ Q′′, and the face F ofoptimal solutions to an LP solved over P ′ ∩ Q′′ can vary for different polyhedra and even for

17

Page 18: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

(a) (b) (c)

P′ ∩ Q′′ ⊃ conv(S(uDW, αDW)) ∩ Q′′ ⊃ F

Q′′

c>

c> c>

Q′′Q′′

P′ P′ P′

F

conv(S(uDW, αDW))

xDW

{s ∈ E | (λDW)s > 0}

F = {xDW}

P′ ∩ Q′′ = conv(S(uDW, αDW)) ∩ Q′′ ⊃ F

conv(S(uDW, αDW))

xDW

{s ∈ E | (λDW)s > 0}

P′ ∩ Q′′ ⊃ conv(S(uDW, αDW)) ∩ Q′′ = F

conv(S(uDW, αDW))

xDW

{s ∈ E | (λDW)s > 0}

F = {xDW}

Figure 8: The relationship of P ′ ∩Q′′, conv(S(uDW , αDW )) ∩Q′′, and the face F .

different objective functions. Figure 8 shows the polyhedra of Example 1 with three differentobjective functions indicated. The convex hull of S(uDW , αDW ) is typically a proper face of P ′,but it is possible for xDW to be an inner point of P ′, in which case we have the following result.

Theorem 4 If xDW is an inner point of P ′, then conv(S(uDW , αDW )) = P ′.

Proof. We prove the contrapositive. Suppose conv(S(uDW , αDW )) is a proper face of P ′. Thenthere exists a facet-defining valid inequality (a, β) ∈ Rn+1 such that conv(S(uDW , αDW )) ⊆ {x ∈Rn | ax = β}. By Theorem 3, xDW ∈ conv(S(uDW , αDW )) and xDW therefore cannot satisfy thedefinition of an inner point.

In this case, illustrated graphically in Figure 8(a) with the polyhedra from Example 1, zDW = zLPand Dantzig-Wolfe decomposition does not improve the bound. All columns of the Dantzig-WolfeLP have reduced cost zero and any member of E can be given positive weight in an optimaldecomposition. A necessary condition for an optimal fractional solution to be an inner point of P ′is that the dual value of the convexity constraint in an optimal solution to the Dantzig-Wolfe LPbe zero. This condition indicates that the chosen relaxation may be too weak.

A second case of potential interest is when F = conv(S(uDW , αDW ))∩Q′′, illustrated graphicallyin Figure 8(b). In this case, all constraints of the Dantzig-Wolfe LP other than the convexityconstraint must have dual value zero, since removing them does not change the optimal solutionvalue. This condition can be detected by examining the objective function values of the membersof E with positive weight in the optimal decomposition. If they are all identical, any such memberthat is contained in Q′′ (if one exists) must be optimal for the original ILP, since it is feasible andhas objective function value equal to zIP . The more typical case, in which F is a proper subset ofconv(S(uDW , αDW )) ∩Q′′, is shown in Figure 8(c).

18

Page 19: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

3.3 Lagrangian Method

The Lagrangian method [22, 14] is a general approach for computing zD that is closely related tothe Dantzig-Wolfe method, but is focused primarily on producing dual solution information. TheLagrangian method can be viewed as a method for producing a particular face of P ′, as in theDantzig-Wolfe method, but no explicit approximation of P ′ is maintained. Although there areimplementations of the Lagrangian method that do produce approximate primal solution informa-tion similar to the solution information that the Dantzig-Wolfe method produces (see Section 3.2),our viewpoint is that the main difference between the Dantzig-Wolfe method and the Lagrangianmethod is the type of solution information they produce. This distinction is important when wediscuss integrated methods in Section 4. When exact primal solution information is not required,faster algorithms for determining the dual solution are possible. By employing a Lagrangian frame-work instead of a Dantzig-Wolfe framework, we can take advantage of this fact.

For a given vector u ∈ Rm′′+ , the Lagrangian relaxation of (1) is given by

zLR(u) = mins∈F ′{c>s+ u>(b′′ −A′′s)}. (43)

It is easily shown that zLR(u) is a lower bound on zIP for any u ≥ 0. The elements of the vectoru are called Lagrange multipliers or dual multipliers with respect to the rows of [A′′, b′′]. Note that(43) is the same subproblem solved in the Dantzig-Wolfe method to generate the most negativereduced cost column. The problem

zLD = maxu∈Rm′′+

{zLR(u)} (44)

of maximizing this bound over all choices of dual multipliers is a dual to (1) called the Lagrangiandual and also provides a lower bound zLD, which we call the LD bound. A vector of multipliers uthat yield the largest bound are called optimal (dual) multipliers.

It is easy to see that zLR(u) is a piecewise linear concave function and can be maximized by anynumber of methods for non-differentiable optimization. In Section 6, we discuss some alternativesolution methods (for a complete treatment, see [34]). In Figure 9 we give an outline of the stepsinvolved in the Lagrangian method. As in Dantzig-Wolfe, the main loop involves updating thedual solution and then generating an improving member of E by solving a subproblem. Unlike theDantzig-Wolfe method, there is no approximation and hence no update step, but the method cannonetheless be viewed in the same frame of reference.

To more clearly see the connection to the Dantzig-Wolfe method, consider the dual of theDantzig-Wolfe LP (33),

zDW = maxα∈R,u∈Rm′′+

{α+ b′′>u | α ≤ (c> − u>A′′)s ∀s ∈ E}. (46)

Letting η = α+ b′′>u and rewriting, we see that

zDW = maxη∈R,u∈Rm′′+

{η | η ≤ (c> − u>A′′)s+ b′′>u ∀s ∈ E} (47)

= maxη∈R,u∈Rm′′+

{mins∈E{(c> − u>A′′)s+ b′′>u}} = zLD. (48)

19

Page 20: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Lagrangian Method

Input: An instance ILP (P, c).Output: A lower bound zLD on the optimal solution value for the instance and adual solution uLD ∈ Rm′′ .

1. Let s0LD ∈ E define some initial extreme point of P ′, u0

LD some initial settingfor the dual multipliers and set t← 0.

2. Master Problem: Using the solution information gained from solving thepricing subproblem, and the previous dual setting utLD, update the dual mul-tipliers ut+1

LD .

3. Subproblem: Call the subroutine OPT (P ′, c> − (utLD)>A′′, (c −(utLD)>A′′)stLD), to solve

ztLD = mins∈F ′{(c> − (utLD)>A′′)s+ b′′>utLD}. (45)

Let st+1LD ∈ E be the optimal solution to this subproblem, if one is found.

4. If a prespecified stopping criterion is met, then output zLD = ztLD and uLD =utLD, otherwise, go to Step 2

Figure 9: Outline of the Lagrangian method

Thus, we have that zLD = zDW and that (44) is another formulation for the problem of calculatingzD. It is also interesting to observe that the set S(utLD, z

tLD − b′′>utLD) is the set of alternative

optimal solutions to the subproblem solved at iteration t in Step 3. The following theorem is acounterpart to Theorem 3 that follows from this observation.

Theorem 5 conv(S(uLD, zLD−b′′>uLD)) is a face of P ′. Also, if F is the face of optimal solutionsto the linear program solved directly over P ′ ∩ Q′′ with objective function vector c, then F ⊆conv(S(uLD, zLD − b′′>uLD)) ∩Q′′.

Again, the proof is similar to that of Theorem 3. This shows that while the Lagrangian methoddoes not maintain an explicit approximation, it does produce a face of P ′ containing the optimalface of solutions to the linear program solved over the approximation P ′ ∩Q′′.

4 Integrated Decomposition Methods

In Section 3, we demonstrated that traditional decomposition approaches can be viewed as utilizingdynamically generated polyhedral information to improve the LP bound by either building an inneror an outer approximation of an implicitly defined polyhedron that approximates P. The choicebetween inner and outer methods is largely an empirical one, but recent computational researchhas favored outer methods. In what follows, we discuss three methods for integrating inner andouter methods. In principle, this is not difficult to do and can result in bounds that are improvedover those achieved by either approach alone.

20

Page 21: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

While traditional decomposition approaches build either an inner or an outer approximation,integrated decomposition methods build both an inner and an outer approximation. These methodsfollow the same basic loop as traditional decomposition methods, except that the master problem isrequired to generate both primal and dual solution information and the subproblem can be eithera separation problem or an optimization problem. The first two techniques we describe integratethe cutting plane method with either the Dantzig-Wolfe method or the Lagrangian method. Thethird technique, described in Section 5, is a cutting plane method that uses an inner approximationto perform separation.

4.1 Price and Cut

The integration of the cutting plane method with the Dantzig-Wolfe method results in a procedurethat alternates between a subproblem that generates improving columns (the pricing subproblem)and a subproblem that generates improving valid inequalities (the cutting subproblem). Hence, wecall the resulting method price and cut. When employed in a branch and bound framework, theoverall technique is called branch, price, and cut. This method has already been studied previouslyby a number of authors [12, 61, 38, 11, 60] and more recently by Aragao and Uchoa [21].

As in the Dantzig-Wolfe method, the bound produced by price and cut can be thought ofas resulting from the intersection of two approximating polyhedra. However, the Dantzig-Wolfemethod required one of these, Q′′, to have a short description. With integrated methods, bothpolyhedra can have descriptions of exponential size. Hence, price and cut allows partial descriptionsof both an inner polyhedron PI and an outer polyhedron PO to be generated dynamically. Tooptimize over the intersection of PI and PO, we use a Dantzig-Wolfe reformulation as in (33),except that the [A′′, b′′] is replaced by a matrix that changes dynamically. The outline of thismethod is shown in Figure 10.

In examining the steps of this generalized method, the most interesting question that arises ishow methods for generating improving columns and valid inequalities translate to this new dynamicsetting. Potentially troublesome is the fact that column generation results in a reduction of thebound ztPC produced by (51), while generation of valid inequalities is aimed at increasing it. Recallagain, however, that while it is the bound ztPC that is directly produced by solving (51), it is thebound ztPC obtained by solving the pricing subproblem that one might claim is more relevant toour goal and this bound can be potentially improved by generation of either valid inequalities orcolumns.

Improving columns can be generated in much the same way as they were in the Dantzig-Wolfemethod. To search for new columns, we simply look for those with negative reduced cost, wherereduced cost is defined to be the usual LP reduced cost with respect to the current reformulation.Having a negative reduced cost is still a necessary condition for a column to be improving. How-ever, it is less clear how to generate improving valid inequalities. Consider an optimal fractionalsolution xtPC obtained by combining the members of E according to weights yielded by the optimaldecomposition λtPC in iteration t. Following a line of reasoning similar to that followed in analyzingthe results of the Dantzig-Wolfe method, we can conclude that xtPC is in fact an optimal solutionto an LP solved directly over PtI ∩PtO with objective function vector c and that therefore, it followsfrom Theorem 1 that any improving inequality must be violated by xtPC . It thus seems sensible toconsider separating xtPC from P. This is the approach taken in the method of Figure 10.

To demonstrate how the price and cut method works, we return to Example 1.

21

Page 22: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Price and Cut Method

Input: An instance ILP (P, c).Output: A lower bound zPC on the optimal solution value for the instance, a pri-mal solution xPC ∈ Rn, an optimal decomposition λPC ∈ RE , a dual solution(uPC , αPC) ∈ Rmt+1, and the inequalities [DPC , dPC ] ∈ Rmt×(n+1).

1. Initialize: Construct an initial inner approximation

P0I = {

∑s∈E0

sλs |∑s∈E0

λs = 1, λs ≥ 0 ∀s ∈ E0, λs = 0 ∀s ∈ E \ E0} ⊆ P ′ (49)

from an initial set E0 of extreme points of P ′ and an initial outer approximation

P0O = {x ∈ Rn | D0x ≥ d0} ⊇ P, (50)

where D0 = A′′ and d0 = b′′, and set t← 0, m0 = m′′.

2. Master Problem: Solve the Dantzig-Wolfe reformulation

ztPC = minλ∈RE+

{c>(∑s∈E

sλs) | Dt(∑s∈E

sλs) ≥ dt,∑s∈E

λs = 1, λs = 0 ∀s ∈ E \ E t}

(51)of the LP over the polyhedron PtI ∩ PtO to obtain the optimal value ztPC ,an optimal primal solution λtPC ∈ RE , an optimal fractional solution xtPC =∑

s∈E s(λtPC)s, and an optimal dual solution (utPC , α

tPC) ∈ Rmt+1.

3. Do either (a) or (b).

(a) Pricing Subproblem and Update: Call the subroutine OPT (P ′, c>−(utPC)>Dt, αtPC), generating a set E of improving members of E withnegative reduced cost (defined in Figure 5). If E 6= ∅, set E t+1 ← E t ∪ Eto form a new inner approximation Pt+1

I . If s ∈ E is the member of Ewith smallest reduced cost, then ztPC = rc(s) +αtPC + (dt)>utPC providesa valid lower bound. Set [Dt+1, dt+1]← [Dt, dt], Pt+1

O ← PtO, mt+1 ← mt,t← t+ 1. and go to Step 2.

(b) Cutting Subproblem and Update: Call the subroutine SEP (P, xtPC)to generate a set of improving valid inequalities [D, d] ∈ Rm×n+1 for P,violated by xtPC . If violated inequalities were found, set [Dt+1, dt+1] ←[Dt dt

D d

]to form a new outer approximation Pt+1

O . Set mt+1 ← mt + m,

E t+1 ← E t, Pt+1I ← PtI , t← t+ 1, and go to Step 2.

4. If E = ∅ and no valid inequalities were found, output the bound zPC = ztPC =

ztPC = c>xtPC , xPC = xtPC , λPC = λtPC , (uPC , αPC) = (utPC , αtPC), and

[DPC , dPC ] = [Dt, dt].

Figure 10: Outline of the price and cut method

22

Page 23: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

(a) (b) (c)

(2,1) (2,1) (2,1)

{s ∈ E | (λ2PC)s > 0}

c>

P

P0O = Q′′

P

P1O = P0

O ∩ {x ∈ Rn | x1 ≥ 3}

P

P2O = P1

O ∩ {x ∈ Rn | x2 ≥ 2}

P0I = conv(E0) ⊂ P′ P1

I = conv(E1) ⊂ P′ P1I = conv(E2) ⊂ P′

x0PC = (2.42, 2.25)

{s ∈ E | (λ0PC)s > 0}

x1PC = (3, 1.5)

{s ∈ E | (λ1PC)s > 0}

x2PC = (3, 2)

Figure 11: Price and cut method (Example 1)

Example 1 (Continued) We pick up the example at the last iteration of the Dantzig-Wolfemethod and show how the bound can be further improved by dynamically generating valid inequal-ities.

Iteration 0. Solving the master problem with E0 = {(4, 1), (5, 5), (2, 1), (3, 4)} and the initial innerapproximation P0

I = conv(E0) yields (λ0PC)(2,1) = 0.58 and (λ0

PC)(3,4) = 0.42, x0PC = (2.42, 2.25),

bound z0PC = z0

PC = 2.42. Next, we solve the cutting subproblem SEP (P, x0PC), generating facet-

defining inequalities of P that are violated by x0PC . One such facet-defining inequality, x1 ≥ 3, is

illustrated in Figure 11(a). We add this inequality to the current set D0 = [A′′, b′′] to form a newouter approximation P1

O, defined by the set D1.

Iteration 1. Solving the new master problem, we obtain an optimal primal solution (λ1PC)(4,1) =

0.42, (λ1PC)(2,1) = 0.42, (λ1

PC)(3,4) = 0.17, x1PC = (3, 1.5), bound z1

PC = 3, as well as an optimaldual solution (u1

PC , α1PC). Next, we consider the pricing subproblem. Since x1

PC is in the interiorof P ′, every extreme point of P ′ has reduced cost 0 by Theorem 4. Therefore, there are no negativereduced cost columns and we switch again to the cutting subproblem SEP (P, x1

PC). As illustratedin Figure 11(b), we find another facet-defining inequality of P violated by x1

PC , x2 ≥ 2. We thenadd this inequality to form D2 and further tighten the outer approximation, now P2

O.

Iteration 2. In the final iteration, we solve the master problem again to obtain (λ2PC)(4,1) = 0.33,

(λ2PC)(2,1) = 0.33, (λ2

PC)(3,4) = 0.33, x2PC = (3, 2), bound z2

PC = 3. Now, since the primal solutionis integral, and is contained in P ′ ∩Q′′, we know that zPC = z2

PC = zIP and we terminate.

Let us now return to the TSP example to further explore the use of the price and cut method.

Example 2 (Continued) As described earlier, application of the Dantzig-Wolfe method alongwith the 1-tree relaxation for the TSP allows us to compute the bound zD obtained by optimizing

23

Page 24: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

over the intersection of the 1-tree polyhedron (the inner polyhedron) with the polyhedron Q′′(the outer polyhedron) defined by constraints (17) and (19). With price and cut, we can furtherimprove the bound by allowing both the inner and outer polyhedra to have large descriptions. Forthis purpose, let us now introduce the well-known comb inequalities [31, 32], which we will generateto improve our outer approximation. A comb is defined by a set H ⊂ V , called the handle and setsT1, T2, ..., Tk ⊂ V , called the teeth, which satisfy

H ∩ Ti 6= ∅ for i = 1, ..., k,

Ti \H 6= ∅ for i = 1, ..., k,

Ti ∩ Tj = ∅ for 1 ≤ i < j ≤ k,

for some odd k ≥ 3. Then, for |V | ≥ 6 the comb inequality,

x(E(H)) +k∑i=1

x(E(Ti)) ≤ |H|+k∑i=1

(|Ti| − 1)− dk/2e (52)

is valid and facet-defining for the TSP. Let the comb polyhedron be defined by constraints (17),(19), and (52).

There are no known efficient algorithms for solving the general facet identification problem forthe comb polyhedron. To overcome this difficulty, one approach is to focus on comb inequalitieswith special forms. One subset of the comb inequalities, known as the blossom inequalities, isobtained by restricting the teeth to have exactly two members. The facet identification for thepolyhedron comprised of the blossom inequalities and constraints (17) and (19) can be solved inpolynomial time, a fact we return to shortly. Another approach is to use heuristic algorithms notguaranteed to find a violated comb inequality when one exists (see [4] for a survey). These heuristicalgorithms could be applied in price and cut as part of the cutting subproblem in Step 3b to improvethe outer approximation.

In Figure 7 of Section 3.2, we showed an optimal fractional solution x that resulted fromthe solution of a Dantzig-Wolfe master problem and the corresponding optimal decomposition,consisting of six 1-trees. In Figure 12, we show the sets H = {0, 1, 2, 3, 6, 7, 9, 11, 12, 15}, T1 ={5, 6}, T2 = {8, 9}, and T3 = {12, 13} forming a comb that is violated by this fractional solution,since

x(E(H)) +

k∑i=1

x(E(Ti)) = 11.3 > 11 = |H|+k∑i=1

(|Ti| − 1)− dk/2e.

Such a violated comb inequality, if found, could be added to the description of the outer polyhedronto improve on the bound zD. This shows the additional power of price and cut over the Dantzig-Wolfe method. Of course, it should be noted that it is also possible to generate such inequalitiesin the standard cutting plane method and to achieve the same bound improvement.

The choice of relaxation has a great deal of effect on the empirical behavior of decompositionalgorithms. In Example 2, we employed an inner polyhedron with integer extreme points. Withsuch a polyhedron, the integrality constraints of the inner polyhedron have no effect and zD = zLP .In Example 3, we consider a relaxation for which the bound zD may be strictly improved over zLPby employing an inner polyhedron that is not integral.

24

Page 25: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

11

0

1213

14

15

1

2

3

4 5

6

7

10

8

9

0.2

0.5

0.5

0.5

0.5

0.2

0.30.7

0.3

0.30.5

0.5

Figure 12: Price and cut method (Example 2)

Example 3 Let G be a graph as defined in Example 2 for the TSP. A 2-matching is a subgraphin which every vertex has degree two. Every TSP tour is hence a 2-matching. The Minimum2-Matching Problem is a relaxation of TSP whose feasible region is described by the degree (17),bound (19), and integrality constraints (20) of the TSP. Interestingly, the 2-matching polyhedron,which is implicitly defined to be the convex hull of the feasible region just described, can also bedescribed by replacing the integrality constraints (20) with the blossom inequalities. Just as theSEC constraints provide a complete description of the 1-tree polyhedron, the blossom inequalities(plus degree and bound) constraints provide a complete description of the 2-matching polyhedron.Therefore, we could use this polyhedron as an outer approximation to the TSP polyhedron. In [50],Muller-Hannemann and Schwartz present several polynomial algorithms for optimizing over the2-matching polyhedron. We can therefore also use the 2-matching relaxation in the context of priceand cut to generate an inner approximation of the TSP polyhedron. Using integrated methods, itwould then be possible to simultaneously build up an outer approximation of the TSP polyhedronconsisting of the SECs (18). Note that this simply reverses the roles of the two polyhedra fromExample 2 and thus would yield the same bound.

Figure 13 shows an optimal fractional solution arising from the solution of the master problemand the 2-matchings with positive weight in a corresponding optimal decomposition. Given thisfractional subgraph, we could employ the separation algorithm discussed in Example 2 of Section3.1 to generate the violated subtour S = {0, 1, 2, 3, 7}.

Another approach to generating improving inequalities in price and cut is to try to take advan-tage of the information contained in the optimal decomposition to aid in the separation procedure.This information, though computed by solving (51) is typically ignored. Consider the fractionalsolution xtPC generated in iteration t of the method in Figure 10. The optimal decomposition forthe master problem in iteration t, λtPC , provides a decomposition of xtPC into a convex combinationof members of E . We refer to elements of E that have a positive weight in this combination asmembers of the decomposition. The following theorem shows how such a decomposition can beused to derive an alternate necessary condition for an inequality to be improving. Because weapply this theorem in a more general context later in the paper, we state it in a general form.

Theorem 6 If x ∈ Rn violates the inequality (a, β) ∈ R(n+1) and λ ∈ RE+ is such that∑

s∈E λs = 1

25

Page 26: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

0

000

1 1 1

1

22 2

2

0.6

3 3 3

3

44

0.2

4

55

0.8

5

666

0.2

77

4

7

5

8 8

6

8

9 9

0.8

9

10

7

10 10

0.8

11 11

8

11

12

9

1212

0.6

13 1313

10

141414

11

1515

0.4

15

0.2

12

0.2

0.2

0.2

13

0.4

0.60.8

14

0.6

0.2

0.2

15

0.2

0.2

0.2

0.8

0.6

(d) λ2 = 0.6

(a) x

(b) λ0 = 0.2 (c) λ1 = 0.2

Figure 13: Finding violated inequalities in price and cut (Example 3)

and x =∑

s∈E sλs, then there must exist an s ∈ E with λs > 0 such that s also violates the inequality(a, β) .

Proof. Let x ∈ Rn and (a, β) ∈ R(n+1) be given such that a>x < β. Also, let λ ∈ RE+ be given

such that∑

s∈E λs = 1 and x =∑

s∈E sλs. Suppose that a>s ≥ β for all s ∈ E with λs > 0.

Since∑

s∈E λs = 1, we have a>(∑

s∈E sλs) ≥ β. Hence, a>x = a>(∑

s∈E sλs) ≥ β, which is acontradiction.

In other words, an inequality can be improving only if it is violated by at least one member ofthe decomposition. If I is the set of all improving inequalities in iteration t, then the followingcorollary is a direct consequence of Theorem 6.

Corollary 4 I ⊆ V = {(a, β) ∈ R(n+1) : a>s < β for some s ∈ E such that (λtPC)s > 0}.

The importance of these results is that in many cases, it is easier to separate members of F ′ fromP than to separate arbitrary real vectors. There are a number of well-known polyhedra for whichthe problem of separating an arbitrary real vector is difficult, but the problem of separating asolution to a given relaxation is easy. This concept is formalized in Section 5 and some examplesare discussed in Section 8. In Figure 14, we propose a new separation procedure that can beembedded in price and cut that takes advantage of this fact. The procedure takes as input anarbitrary real vector x that has been previously decomposed into a convex combination of vectorswith known structure. In price and cut, the arbitrary real vector is xtPC and it is decomposed intoa convex combination of members of E by solving the master problem (51). Rather than separating

26

Page 27: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Separation using a Decomposition

Input: A decomposition λ ∈ RE of x ∈ Rn.Output: A set [D, d] of potentially improving inequalities.

1. Form the set D = {s ∈ E | λs > 0}.

2. For each s ∈ D, call the subroutine SEP (P, s) to obtain a set [D, d] of violatedinequalities.

3. Let [D, d] be composed of the inequalities found in Step 2 that are also violatedby x, so that Dx < d.

4. Return [D, d] as the set of potentially improving inequalities.

Figure 14: Solving the cutting subproblem with the aid of a decomposition

xtPC directly, the procedure consists of separating each one of the members of the decompositionin turn, then checking each inequality found for violation against xtPC .

The running time of this procedure depends in part on the cardinality of the decomposition.Caratheodory’s Theorem assures us that there exists a decomposition with less than or equal todim(PtI) + 1 members. Unfortunately, even if we limit our search to a particular known class ofvalid inequalities, the number of such inequalities violated by each member of D in Step 2 may beextremely large and these inequalities may not be violated by xtPC (such an inequality cannot beimproving). Unless we enumerate every inequality in the set V from Corollary 4, either implicitlyor explicitly, the procedure does not guarantee that an improving inequality will be found, evenif one exists. In cases where it is possible to examine the set V in polynomial time, the worst-case complexity of the entire procedure is polynomially equivalent to that of optimizing over P ′.Obviously, it is unlikely that the set V can be examined in polynomial time in situations whenseparating xtPC is itself an NP-complete problem. In such cases, the procedure to select inequalitiesthat are likely to be violated by xtPC in Step 2 is necessarily a problem-dependent heuristic. Theeffectiveness of such heuristics can be improved in a number of ways, some of which are discussedin [57].

Note that members of the decomposition in iteration t must belong to the set S(utPC , αtPC), as

defined by (40). It follows that the convex hull of the decomposition is a subset of conv(S(utPC , αtPC))

that contains xtPC and can be thought of as a surrogate for the face of optimal solutions to an LPsolved directly over PtI ∩PtO with objective function vector c. Combining this corollary with Theo-rem 1, we conclude that separation of S(utPC , α

tPC) from P is a sufficient condition for an inequality

to be improving. Although this sufficient condition is difficult to verify in practice, it does provideadditional motivation for the method described in Figure 14.

Example 1 (Continued) Returning to the cutting subproblem in iteration 0 of the price and cutmethod, we have a decomposition x0

PC = (2.42, 2.25) = 0.58(2, 1)+0.42(3, 4), as depicted in Figure11(a). Now, instead of trying to solve the subproblem SEP (P, x0

PC), we instead solve SEP (P, s),for each s ∈ D = {(2, 1), (3, 4)}. In this case, when solving the separation problem for s = (2, 1),we find the same facet-defining inequality of P as we did by separating x0

PC directly.

27

Page 28: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

15

14

0

12

0

13

11

1 1

22

3 3

44 5 5

66

7 7

8

10

8

99

10

11

1213

14

15

0.2

0.2

0.3

0.3

0.70.5

0.5

0.5

0.5

0.5

0.5

0.3

(a) λ0(a) x

Figure 15: Using the optimal decomposition to find violated inequalities in price and cut (Example2)

Similarly, in iteration 1, we have a decomposition of x2PC = (3, 1.5) into a convex combination of

D = {(4, 1), (2, 1), (3, 4)}. Clearly, solving the separation problem for either (2, 1) or (4, 1) producesthe same facet-defining inequality as with the original method.

Example 2 (Continued) Returning again to Example 2, recall the optimal fractional solutionand the corresponding optimal decomposition arising during solution of the TSP by the Dantzig-Wolfe method in Figure 7. Figure 12 shows a comb inequality violated by this fractional solution.By Theorem 6, at least one of the members of the optimal decomposition shown in Figure 7 mustalso violate this inequality. In fact, the member with index 0, also shown in Figure 15, is the onlysuch member. Note that the violation is easy to discern from the structure of this integral solution.Let x ∈ {0, 1}E be the incidence vector of a 1-tree. Consider a subset H of V whose inducedsubgraph in the 1-tree is a path with edge set P . Consider also an odd set O of edges of the 1-treeof cardinality at least 3 and disjoint from P , such that each edge has one endpoint in H and oneendpoint in V \ H. Taking the set H to be the handle and the endpoints of each member of Oto be the teeth, it is easy to verify that the corresponding comb inequality will be violated by the1-tree, since

x(E(H)) +k∑i=1

x(E(Ti)) = |H| − 1 +k∑i=1

(|Ti| − 1) > |H|+k∑i=1

(|Ti| − 1)− dk/2e.

Hence, searching for such configurations in the members of the decomposition, as suggested inthe procedure of Figure 14, may lead to the discovery of comb inequalities violated by the optimalfractional solution. In this case, such a configuration does in fact lead to discovery of the previouslyindicated comb inequality. Note that we have restricted ourselves in the above discussion to thegeneration of blossom inequalities. The teeth, as well as the handles can have more general formsthat may lead to the discovery of more general forms of violated combs.

Example 3 (Continued) Returning now to Example 3, recall the optimal fractional solutionand the corresponding optimal decomposition, consisting of the 2-matchings shown in Figure 13.

28

Page 29: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

00

11

22

0.6

33

4

0.2

5

0.8

6

0.2

7

4

5

8

6

9

0.8

7

10

0.8

8

11

9

12

0.6

13

10

1411

15

0.4

0.2

12

0.2

0.2

0.2

13

0.4

0.60.8

14

0.6

0.2

0.2

15

0.2

0.2

0.2

0.8

0.6

(a) x (b) λ2

Figure 16: Using the optimal decomposition to find violated inequalities in price and cut (Example3)

Previously, we produced a set of vertices defining a SEC violated by the fractional point by using aminimum cut algorithm with the optimal fractional solution as input. Now, let us consider applyingthe procedure of Figure 14 by examining the members of the decomposition in order to discoveredinequalities violated by the optimal fractional solution. Let x ∈ {0, 1}E be the incidence vector of a2-matching. If the corresponding subgraph does not form a tour, then it must be disconnected. Thevertices corresponding to any connected component thus define a violated SEC. By determiningthe connected components of each member of the decomposition, it is easy to find violated SECs.In fact, for any 2-matching, every component of the 2-matching forms a SEC that is violated byexactly 1. For the 2-matching corresponding to s, we have x(E(S)) = |S| > |S| − 1. Figure 16(b)shows the third member of the decomposition along with a violated SEC defined by one of itscomponents. This same SEC is also violated by the optimal fractional solution.

There are many variants of the price and cut method shown in Figure 10. Most significantis the choice of which subproblem to execute during Step 3. It is easy to envision a number ofheuristic rules for deciding this. For example, one obvious rule is to continue generating columnsuntil no more are available and then switch to valid inequalities for one iteration, then generatecolumns again until none are available. This can be seen as performing a “complete” dual solutionupdate before generating valid inequalities. Further variants can be obtained by not insisting ona “complete” dual update before solving the pricing problem [29, 17]. This rule could easily beinverted to generate valid inequalities until no more are available and then generate columns. Ahybrid rule in which some sort of alternation occurs is a third option. The choice between theseoptions is primarily empirical.

4.2 Relax and Cut

Just as with the Dantzig-Wolfe method, the Lagrangian method of Figure 9 can be integrated withthe cutting plane method to yield a procedure several authors have termed relax and cut. This isdone in much the same fashion as in price and cut, with a choice in each iteration between solving

29

Page 30: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

a pricing subproblem and a cutting subproblem. In each iteration that the cutting subproblem issolved, the generated valid inequalities are added to the description of the outer polyhedron, whichis explicitly maintained as the algorithm proceeds. As with the traditional Lagrangian method, noexplicit inner polyhedron is maintained, but the algorithm can again be seen as one that computesa face of the implicitly defined inner polyhedron that contains the optimal face of solutions to alinear program solved over the intersection of the two polyhedra. When employed within a branchand bound framework, we call the overall method branch, relax, and cut.

An outline of the relax and cut method is shown in Figure 17. The question again arises as tohow to ensure that the inequalities being generated in the cutting subproblem are improving. Inthe case of the Lagrangian method, this is a much more difficult issue since we cannot assume theavailability of the same primal solution information available within price and cut. Furthermore,we cannot verify the condition of Corollary 1, which is the best available necessary condition for aninequality to be improving. Nevertheless, some primal solution information is always available inthe form of the solution stRC to the last pricing subproblem that was solved. Intuitively, separatingstRC makes sense since the infeasibilities present in stRC may possibly be removed through theaddition of valid inequalities violated by stRC .

As with both the cutting plane and price and cut methods, the difficulty is that the validinequalities generated by separating stRC from P may not be improving, as Guignard first observedin [33]. To deepen understanding of the potential effectiveness of the valid inequalities generated, wefurther examine the relationship between stRC and xtPC by recalling again the results from Section3.2. Consider the set S(utRC , z

tRC), where ztRC is obtained by solving the pricing subproblem

(54) from Figure 17 and the set S(·, ·) is as defined in (40). In each iteration where the pricingsubproblem is solved, st+1

RC is a member of S(utRC , ztRC). In fact, S(utRC , z

tRC) is exactly the set of

alternative solutions to this pricing subproblem. In price and cut, a number of members of thisset are available, one of which must be violated in order for a given inequality to be improving.This yields a verifiable necessary condition for a generated inequality to be improving. Relax andcut, in its most straightforward incarnation, produces one member of this set. Even if improvinginequalities exist, it is possible that none of them are violated by the member of S(utRC , z

tRC) so

produced, especially if it would have had a small weight in the optimal decomposition produced bythe corresponding iteration of price and cut.

It is important to note that by keeping track of the solutions to the Lagrangian subproblem thatare produced while solving the Lagrangian dual, one can approximate the optimal decompositionand the optimal fractional solution produced by solving (51). This is the approach taken by thevolume algorithm [9] and a number of other subgradient-based methods. As in price and cut, whenthis fractional solution is an inner point of P ′, all members of F ′ are alternative optimal solutionsto the pricing subproblem and the bound is not improved over what the cutting plane methodalone would produce. In this case, solving the cutting subproblem to obtain additional inequalitiesis unlikely to yield further improvement.

As with price and cut, there are again many variants of the algorithm shown in Figure 17,depending on the choice of subproblem to execute at each step. One such variant is to alternatebetween each of the subproblems, first solving one and then the other [45]. In this case, theLagrangian dual is not solved to optimality before solving the cutting subproblem. Alternatively,another approach is to solve the Lagrangian dual all the way to optimality before generating validinequalities. Again, the choice is primarily empirical.

30

Page 31: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Relax and Cut Method

Input: An instance ILP (P, c).Output: A lower bound zRC on the optimal solution value for the instance and adual solution uRC ∈ Rmt .

1. Let s0RC ∈ E define some initial extreme point of P ′ and construct an initial

outer approximation

P0O = {x ∈ Rn | D0x ≥ d0} ⊇ P, (53)

where D0 = A′′ and d0 = b′′. Let u0RC ∈ Rm′′ be some initial set of dual

multipliers associated with the constraints [D0, d0]. Set t← 0 and mt = m′′.

2. Master Problem: Using the solution information gained from solving thepricing subproblem, and the previous dual solution utRC , update the dualsolution (if the pricing problem was just solved) or initialize the new dualmultipliers (if the cutting subproblem was just solved) to obtain ut+1

RC ∈ Rmt .

3. Do either (a) or (b).

(a) Pricing Subproblem: Call the subroutine OPT (P ′, c− (utRC)>Dt, (c−(utRC)>Dt)stRC) to obtain

ztRC = mins∈F ′{(c> − (utRC)Dt)s+ dt(utRC)}. (54)

Let st+1RC ∈ E be the optimal solution to this subproblem. Set

[Dt+1, dt+1] ← [Dt, dt], Pt+1O ← PtO, mt+1 ← mt, t ← t + 1, and go

to Step 2.

(b) Cutting Subproblem: Call the subroutine SEP (P, stRC) to generatea set of improving valid inequalities [D, d] ∈ Rm×n+1 for P, violated bystRC . If violated inequalities were found, set [Dt+1, dt+1] ←

[Dt dt

D d

]to

form a new outer approximation Pt+1O . Set mt+1 ← mt+ m, st+1

RC ← stRC ,t← t+ 1, and go to Step 2.

4. If a prespecified stopping criterion is met, then output zRC = ztRC and uRC =utRC .

5. Otherwise, go to Step 2.

Figure 17: Outline of the relax and cut method

31

Page 32: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

5 Solving the Cutting Subproblem

In this section, we formalize some notions that have been introduced in our examples and providemore details regarding how the cutting subproblem is solved in practice in the context of thevarious methods we have outlined. We review the well-known template paradigm for separation andintroduce a new concept called structured separation. Finally, we describe a separation algorithmcalled decompose and cut that is closely related to the integrated decomposition methods we havealready described and utilizes several of the concepts introduced earlier.

5.1 The Template Paradigm

The ability to generate valid inequalities for P violated by a given real vector is a crucial step inmany of the methods discussed in this paper. Ideally, we would be able to solve the general facetidentification problem for P, allowing us to generate a violated valid inequality whenever one exists.This is clearly not practical in most cases, since the complexity of this problem is the same as thatof solving the original ILP. In practice, the subproblem SEP (P, xtCP ) in Step 3 of the cutting planemethod pictured in Figure 2 is usually solved by dividing the valid inequalities for P into templateclasses with known structure. Procedures are then designed and executed for identifying violatedmembers of each class individually.

A template class (or simply class) of valid inequalities for P is a set of related valid inequal-ities that describes a polyhedron containing P, so we can identify each class with its associatedpolyhedron. In Example 2, we described two well-known classes of valid inequalities for the TSP,the subtour elimination constraints and the comb inequalities. Both classes have an identifiablecoefficient structure and describe polyhedra containing P. Consider a polyhedron C described by aclass of valid inequalities for P. The separation problem for the class C of valid inequalities for P isdefined to be the facet identification problem over the polyhedron C. In other words, the separationproblem for a class of valid inequalities depends on the form of the inequality and is independent ofthe polyhedron P. It follows that the worst case running time for solving the separation problemis also independent of P. In particular, the separation problem for a particular class of inequalitiesmay be much easier to solve than the general facet identification problem for P. Therefore, inpractice, the separation problem is usually attempted over “easy” classes first, and more difficultclasses are only attempted when needed. In the case of the TSP, the separation problem for theSECs is solvable in polynomial time, whereas there is no known efficient algorithm for solving theseparation problem for comb inequalities. In general, the intersection of the polyhedra associatedwith the classes of inequalities for which the separation problem can be reasonably solved is notequal to P.

5.2 Separating Solutions with Known Structure

In many cases, the complexity of the separation problem is also affected by the structure of the realvector being separated. In Section 4, we informally introduced the notion that a solution vectorwith known structure may be easier to separate from a given polyhedron than an arbitrary one andillustrated this phenomenon in Examples 2 and 3. This is a concept called structured separation thatarises quite frequently in the solution of combinatorial optimization problems where the originalformulation is of exponential size. When using the cutting plane method to solve the LP relaxationof the TSP, for example, as described in Example 2, we must generate the SECs dynamically.

32

Page 33: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

It is thus possible that the intermediate solutions are integer-valued, but nonetheless not feasiblebecause they violate some SEC that is not present in the current approximation. When the currentsolution is optimal, however, it is easy to determine whether it violates a SEC by simply examiningthe connected components of the underlying support graph, as described earlier. This process canbe done in O(|V | + |E|) time. For an arbitrary real vector, the separation problem for SECs ismore difficult, taking O(|V |4) time.

It is also frequently the case that when applying a sequence of separation routines for progres-sively more difficult classes of inequalities, routines for the more difficult classes assume implicitlythat the solution to be separated satisfies all inequalities of the the easier classes. In the case ofthe TSP, for instance, any solution passed to the subroutine for separating the comb inequalities isgenerally assumed to satisfy the degree and subtour elimination constraints. This assumption canallow the separation algorithms for subsequent classes to be implemented more efficiently.

For the purposes of the present work, our main concern is with separating solutions that areknown to be integral, in particular, members of F ′. In our framework, the concept of structuredseparation is combined with the template paradigm in specifying template classes of inequalitiesfor which separation of integral solutions is much easier, in a complexity sense, than separationof arbitrary real vectors over that same class. A number of examples of problems and classes ofvalid inequalities for which this situation occurs are examined in Section 8. We now examine aseparation paradigm called decompose and cut that can take advantage of our ability to easilyseparate solutions with structure.

5.3 Decompose and Cut

The use of a decomposition to aid in separation, as is described in the procedure of Figure 14, iseasy to extend to a traditional branch and cut framework using a technique we call decompose andcut, originally proposed in [56] and further developed in [39] and [57]. Suppose now that we aregiven an optimal fractional solution xtCP obtained during iteration t of the cutting plane methodand suppose that for a given s ∈ F ′, we can determine effectively whether s ∈ F and if not, generatea valid inequality (a, β) violated by s. By first decomposing xtCP (i.e., expressing xtCP as a convexcombination of members of E ⊆ F ′) and then separating each member of this decomposition fromP in the fashion described in Figure 14, we may be able to find valid inequalities for P that areviolated by xtCP .

The difficult step is finding the decomposition of xtCP . This can be accomplished by solving alinear program whose columns are the members of E , as described in Figure 18. This linear programis reminiscent of (33) and in fact can be solved using an analogous column-generation scheme, asdescribed in Figure 19. This scheme can be seen as the “inverse” of the method described inSection 4.1, since it begins with the fractional solution xtCP and tries to compute a decomposition,instead of the other way around. By the equivalence of optimization and facet identification, wecan conclude that the problem of finding a decomposition of xtCP is polynomially equivalent to thatof optimizing over P ′.

Once the decomposition is found, it can be used as before to locate a violated valid inequality.In contrast to price and cut, however, it is possible that xtCP 6∈ P ′. This could occur, for instance, ifexact separation methods for P ′ are too expensive to apply consistently. In this case, it is obviouslynot possible to find a decomposition in Step 2 of Figure 18. The proof of infeasibility for the linearprogram (55), however, provides an inequality separating xtCP from P ′ at no additional expense.Hence, even if we fail to find a decomposition, we still find an inequality valid for P and violated

33

Page 34: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Separation in Decompose and Cut

Input: x ∈ RnOutput: A valid inequality for P violated by x, if one is found.

1. Apply standard separation techniques to separate x. If one of these returns aviolated inequality, then STOP and output the violated inequality.

2. Otherwise, solve the linear program

maxλ∈RE+

{0>λ |∑s∈E

sλs = x,∑s∈E

λs = 1}, (55)

as in Figure 19.

3. The result of Step 2 is either (1) a subset D of members of E participating in aconvex combination of x, or (2) a valid inequality (a, β) for P that is violatedby x. In the first case, go to Step 4. In the second case, STOP and outputthe violated inequality.

4. Attempt to separate each member of D from P. For each inequality violatedby a member of D, check whether it is also violated by x. If an inequalityviolated by x is encountered, STOP and output it.

Figure 18: Separation in the decompose and cut method

Column Generation in Decompose and Cut

Input: x ∈ RnOutput: Either (1) a valid inequality for P violated by x; or (2) a subset D of E anda vector λ ∈ RE+ such that

∑s∈D λss = x and

∑s∈D λs = 1.

2.0 Generate an initial subset E0 of E and set t← 0.

2.1 Solve (55), replacing E by E t. If this linear program is feasible, then theelements of E t corresponding to the nonzero components of λ, the currentsolution, comprise the set D, so STOP.

2.2 Otherwise, let (a, β) be a valid inequality for conv(E t) violated by x (i.e., theproof of infeasibility). Solve OPT (P ′, a, β) and let E be the resulting set ofsolutions. If E 6= ∅, then set E t+1 ← E t∪E , t→ t+1, and go to 2.1. Otherwise,(a, β) is an inequality valid for P ′ ⊇ P and violated by x, so STOP.

Figure 19: Column generation for the decompose and cut method

34

Page 35: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

by xtCP . This idea was originally suggested in [56] and was further developed in [39]. A similarconcept was also discovered and developed independently by Applegate, et al. [3].

Applying decompose and cut in every iteration as the sole means of separation is theoreticallyequivalent to price and cut. In practice, however, the decomposition is only computed when needed,i.e., when less expensive separation heuristics fail to separate the optimal fractional solution. Thiscould give decompose and cut an advantage in terms of computational efficiency. In other respects,the computations performed in each method are similar.

6 Solving the Master Problem

The choice of a proper algorithm for solving the master problem is important for these methods,both because a significant portion of the computational effort is spent solving the master problemand because the solver must be capable of returning the solution information required by themethod. In this section, we would like to briefly give the reader a taste for the issues involved andsummarize the existing methodology. The master problems we have discussed are linear programs,or can be reformulated as linear programs. Hence, one option for solving them is to use eithersimplex or interior point methods. In the case of solving a Lagrangian dual, subgradient methodsmay also be employed.

Simplex methods have the advantage of providing accurate primal solution information. Theyare therefore well-suited for algorithms that utilize primal solution information, such as price andcut. The drawback of these methods is that updates to the dual solution at each iteration arerelatively expensive. In their most straightforward implementations, they also tend to convergeslowly when used with column generation. This is primarily due to the fact that they producebasic (extremal) dual solutions, which tend to change substantially from one iteration to the next,causing wide oscillations in the input to the column-generation subproblem. This problem can beaddressed by implementing one of a number of stabilization methods that prevent the dual solutionfrom changing “too much” from one iteration to the next (for a survey, see [42]).

Subgradient methods, on the other hand, do not produce primal solution information in theirmost straightforward form, so they are generally most appropriate for Lagrangian methods suchas relax and cut. It is possible, however, to obtain approximate primal solutions from variants ofsubgradient such as the volume algorithm [9]. Subgradient methods also have convergence issueswithout some form of stabilization. A recent class of algorithms that has proven effective in thisregard is bundle methods [18].

Interior point methods may provide a middle ground by providing accurate primal solutioninformation and more stable dual solutions [58, 28]. In addition, hybrid methods that alternatebetween simplex and subgradient methods for updating the dual solution have also shown promise[10, 36].

7 Software

The theoretical and algorithmic framework proposed in Sections 3–5 lends itself nicely to a wide-ranging and flexible generic software framework. All of the techniques discussed can be implementedby combining a set of basic algorithmic building blocks. DECOMP is a C++ framework designedwith the goal of providing a user with the ability to easily utilize various traditional and integrateddecomposition methods while requiring only the provision of minimal problem-specific algorithmic

35

Page 36: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

components [25]. With DECOMP, the majority of the algorithmic structure is provided as part ofthe framework, making it easy to compare various algorithms directly and determine which optionis the best for a given problem setting. In addition, DECOMP is extensible—each algorithmiccomponent can be overridden by the user, if they so wish, in order to develop sophisticated variantsof the aforementioned methods.

The framework is divided into two separate user interfaces, an applications interface DecompApp,in which the user must provide implementations of problem-specific methods (e.g., solvers for thesubproblems), and an algorithms interface DecompAlgo, in which the user can modify DECOMP’sinternal algorithms, if desired. A DecompAlgo object provides implementations of all of the methodsdescribed in Sections 3 and 4, as well as options for solving the master problem, as discussed inSection 6. One important feature of DECOMP is that the problem is always represented in theoriginal space, rather than in the space of a particular reformulation. The user has only to providesubroutines for separation and column generation in the original space without considering theunderlying method. The framework performs all of the necessary bookkeeping tasks, includingincluding automatic reformulation in the Dantzig-Wolfe master, constraint dualization for relaxand cut, cut and variable pool management, as well as, row and column expansion.

In order to develop an application, the user must provide implementations of the following twomethods.

• DecompApp::createCore(). The user must define the initial set of constraints [A′′, b′′].

• DecompApp::solveRelaxedProblem(). The user must provide a solver for the relaxed prob-lem OPT (P ′, c, U) that takes a cost vector c ∈ Rn as its input and returns a set of solutionsas DecompVar objects. Alternatively, the user has the option to provide the inequality set[A′, b′] and solve the relaxed problem using the built-in ILP solver.

If the user wishes to invoke the traditional cutting plane method using problem-specific methods,then the following method must also be implemented.

• DecompApp::generateCuts(x). A method for solving the separation problem SEP (P, x),given an arbitrary real vector x ∈ Rn, which returns a set of DecompCut objects.

Alternatively, various generic separation algorithms are also provided. The user might also wishto implement separation routines specifically for members of F ′ that can take advantage of thestructure of such solutions, as was described in Section 5.

• DecompApp::generateCuts(s). A method for solving the separation problem SEP (P, s),given s ∈ F ′, which returns a set of DecompCut objects.

At a high level, the main loop of the base algorithm provided in DecompAlgo follows the paradigmdescribed earlier, alternating between solving a master problem to obtain solution information,followed by a subproblem to generate new polyhedral information. Each of the methods describedin this paper have its own separate interface derived from DecompAlgo. For example, the base classfor the price and cut method is DecompAlgo::DecompAlgoPC. In this manner, the user can overridea specific subroutine common to all methods (in DecompAlgo) or restrict it to a particular method.

36

Page 37: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

8 Applications

In this section, we further illustrate the concepts presented with three more examples. We focus hereon the application of integrated methods, a key component of which is the paradigm of structuredseparation introduced in Section 5. For each example, we discuss three key polyhedra: (1) anoriginal ILP defined by a polyhedron P and associated feasible set F = P ∩ Zn; (2) a relaxationof the original ILP with feasible set F ′ ⊇ F such that effective optimization over the polyhedronPI = conv(F ′) is possible; and (3) a polyhedron PO, such that F = PI ∩PO ∩Zn. In each case, thepolyhedron PO is comprised of a known class or classes of valid inequalities that could be generatedduring execution of the cutting subproblem of one of the integrated methods discussed in Section4. As before, PI is a polyhedron with an inner description generated dynamically through thesolution of an optimization problem, while PO is a polyhedron with an outer description generateddynamically through the solution of a separation problem. We do not discuss standard methods ofsolving the separation problem for PO, i.e., unstructured separation, as these are well-covered in theliterature. Instead, we focus here on problems and classes of valid inequalities for which structuredseparation, i.e., separation of a member of F ′, is much easier than unstructured separation. Anumber of ILPs that have appeared in the literature have relaxations and associated classes of validinequalities that fit into this framework, such as the Generalized Assignment Problem [54], the Edge-Weighted Clique Problem [37], the Knapsack Constrained Circuit Problem [41], the RectangularPartition Problem [16], the Linear Ordering Problem [15], and the Capacitated Minimum SpanningTree Problem [24].

8.1 Vehicle Routing Problem

We first consider the Vehicle Routing Problem (VRP) introduced by Dantzig and Ramser [20]. Inthis NP-hard optimization problem, a fleet of k vehicles with uniform capacity C must serviceknown customer demands for a single commodity from a common depot at minimum cost. LetV = {1, . . . , |V |} index the set of customers and let the depot have index 0. Associated with eachcustomer i ∈ V is a demand di. The cost of travel from customer i to j is denoted cij and weassume that cij = cji > 0 if i 6= j and cii = 0.

By constructing an associated complete undirected graph G with vertex set N = V ∪ {0} andedge set E, we can formulate the VRP as an integer program. A route is a set of vertices R ={i1, i2, . . . , im} such that the members of R are distinct. The edge set of R is ER = {{ij , ij+1} | j ∈0, . . . ,m}, where i0 = im+1 = 0. A feasible solution is then any subset of E that is the union ofthe edge sets of k disjoint routes Ri, i ∈ [1..k], each of which satisfies the capacity restriction, i.e.,∑

j∈Ri dj ≤ C, ∀i ∈ [1..k]. Each route corresponds to a set of customers serviced by one of the kvehicles. To simplify the presentation, let us define some additional notation.

By associating a variable with each edge in the graph, we obtain the following formulation of

37

Page 38: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

this ILP [40]:

min∑e∈E

cexe,

x(δ({0})) = 2k, (56)

x(δ({v})) = 2 ∀v ∈ V, (57)

x(δ(S)) ≥ 2b(S) ∀S ⊆ V, |S| > 1, (58)

xe ∈ {0, 1} ∀e ∈ E(V ), (59)

xe ∈ {0, 1, 2} ∀e ∈ δ(0). (60)

Here, b(S) represents a lower bound on the number of vehicles required to service the set of cus-tomers S. Inequalities (56) ensure that there are exactly k vehicles, each departing from andreturning to the depot, while inequalities (57) require that each customer must be serviced byexactly one vehicle. Inequalities (58), known as the generalized subtour elimination constraints(GSECs) can be viewed as a generalization of the subtour elimination constraints from TSP, andenforce connectivity of the solution, as well as ensuring that no route has total demand exceedingcapacity C. For ease of computation, we can define b(S) =

⌈(∑i∈S di

)/C⌉, a trivial lower bound

on the number of vehicles required to service the set of customers S.The set of feasible solutions to the VRP is

F = {x ∈ RE | x satisfies (56)− (60)}

and we call P = conv(F) the VRP polyhedron. Many classes of valid inequalities for the VRPpolyhedron have been reported in the literature (see [51] for a survey). Significant effort has beendevoted to developing efficient algorithms for separating an arbitrary fractional point using theseclasses of inequalities (see [46] for recent results).

We concentrate here on the separation of GSECs. The separation problem for GSECs was shownto be NP-complete by Harche and Rinaldi (see [5]), even when b(S) is taken to be

⌈(∑i∈S di

)/C⌉.

In [46], Lysgaard, et al. review heuristic procedures for generating violated GSECs. AlthoughGSECs are part of the formulation presented above, there are exponentially many of them, sowe generate them dynamically. We discuss three relaxations of the VRP: the Multiple TravelingSalesman Problem, the Perfect b-Matching Problem, and the Minimum Degree-constrained k-TreeProblem. For each of these alternatives, violation of GSECs by solutions to the relaxation can beeasily discerned.

Perfect b-Matching Problem. With respect to the graph G, the Perfect b-Matching Problemis to find a minimum weight subgraph of G such that x(δ(v)) = bv ∀v ∈ V . This problem can beformulated by dropping the GSECs from the VRP formulation, resulting in the feasible set

F ′ = {x ∈ RE | x satisfies (56), (57), (59), (60)}.

In [50], Muller-Hannemann and Schwartz, present several fast polynomial algorithms for solvingb-Matching. The polyhedron PO consists of the GSECs (58) in this case.

38

Page 39: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

In [49], Miller uses the b-Matching relaxation to solve the VRP by branch, relax, and cut. Hesuggests generating GSECS violated by b-matchings as follows. Consider a member s of F ′ and itssupport graph Gs (a b-Matching). If Gs is disconnected, then each component immediately inducesa violated GSEC. On the other hand, if Gs is connected, we first remove the edges incident to thedepot vertex and find the connected components, which comprise the routes described earlier. Toidentify a violated GSEC, we compute the total demand of each route, checking whether it exceedscapacity. If not, the solution is feasible for the original ILP and does not violate any GSECs. Ifso, the set S of customers on any route whose total demand exceeds capacity induces a violatedGSEC. This separation routine runs in O(|V |+ |E|) time and can be used in any of the integrateddecomposition methods previously described. Figure 20(a) shows an example vector that couldarise during execution of either price and cut or decompose and cut, along with a decompositioninto a convex combination of two b-Matchings, shown in Figures 20(b) and 20(c). In this example,the capacity C = 35 and by inspection we find a violated GSEC in the second b-Matching (c)with S equal to the marked component. This inequality is also violated by the optimal fractionalsolution, since x(δ(S)) = 3.0 < 4.0 = 2b(S).

Minimum Degree-constrained k-Tree Problem. A k-tree is defined as a spanning subgraphof G that has |V |+k edges (recall that G has |V |+1 vertices). A degree-constrained k-tree (k-DCT),as defined by Fisher in [23], is a k-tree with degree 2k at vertex 0. The Minimum k-DCT Problemis that of finding a minimum cost k-DCT, where the cost of a k-DCT is the sum of the costs onthe edges present in the k-DCT. Fisher [23] introduced this relaxation as part of a Lagrangianrelaxation-based algorithm for solving the VRP.

The k-DCT polyhedron is obtained by first adding the redundant constraint

x(E) = |V |+ k, (61)

then deleting the degree constraints (57), and finally, relaxing the capacity to C =∑

i∈S di. Relax-ing the capacity constraints gives b(S) = 1 for all S ⊆ V , and replaces the set of constraints (58)with ∑

e∈δ(S)

xe ≥ 2,∀S ⊆ V, |S| > 1. (62)

The feasible region of the Minimum k-DCT Problem is then

F ′ = {x ∈ RE | x satisfies (56), (58), (59), (61)}.

This time, the polyhedron PO is comprised of the constraints (57) and the GSECs (58). Since theconstraints (57) can be represented explicitly, we focus again on generation of violated GSECs. In[62], Wei and Yu give a polynomial algorithm for solving the Minimum k-DCT Problem that runsin O(|V |2 log |V |) time. In [48], Martinhon et al. study the use of the k-DCT relaxation for theVRP in the context branch, relax, and cut. Again, consider separating a member s of F ′ fromthe polyhedron defined by all GSECS. It is easy to see that for GSECs, an algorithm identical tothat described above can be applied. Figure 20(a) also shows a vector that could arise during theexecution of either the price and cut or decompose and cut algorithms, along with a decompositioninto a convex combination of four k-DCTs, shown in Figures 20(d) through 20(g). Removing thedepot edges and checking each component’s demand, we easily identify the violated GSEC shownin Figure 20(g).

39

Page 40: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

00

0

00

0

0

2

2

22

2

2

2

7

7

7

7

7

7

7

5

5

5

5

5

5

5

8

8

8

8

8

8

8

7

7

7

77

7

7

10

10

10

10

10

10

10

4

4

4

4

4

4

4

0.5

11

11

11

11

0.5

11

11

5

0.5

5

5

5

5

5

11

1

1

1

1

1

1

5

13

13

13

13

1

13

13

5

5

5

55

5

13

4

4

4

4

5

4

4

1

1

1

1

1

0.5

1

0.5

1

1

1

1

1

1

4

2

2

2

2

0.5

2

2

1

12

0.5

0.5

(g) k-DCT λ4 = 14

(a) x

(b) b-Matching λ1 = 12

(c) b-Matching λ2 = 12

(f) k-DCT λ3 = 14

(e) k-DCT λ2 = 14

(d) k-DCT λ1 = 14

Figure 20: Example of a decomposition into b-Matchings and k-DCTs

40

Page 41: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

Multiple Traveling Salesman Problem. The Multiple Traveling Salesman Problem (k-TSP)is an uncapacitated version of the VRP obtained by adding the degree constraints to the k-DCTpolyhedron. The feasible region of the k-TSP is

F ′ = {x ∈ RE | x satisfies (56), (57), (59), (60), (62)}.

Although the k-TSP is an NP-hard optimization problem, small instances can be solved effectivelyby transformation into an equivalent TSP obtained by adjoining to the graph k − 1 additionalcopies of vertex 0 and its incident edges. In this case, the polyhedron PO is again comprised solelyof the GSECs (58). In [57], Ralphs et al. report on an implementation of branch, decompose andcut using the k-TSP as a relaxation.

8.2 Three-Index Assignment Problem

The Three-Index Assignment Problem (3AP) is that of finding a minimum-weight clique cover ofthe complete tri-partite graph Kn,n,n. Let I, J and K be three disjoint sets with |I| = |J | = |K| = nand set H = I × J ×K. 3AP can be formulated as the following binary integer program:

min∑

(i,j,k)∈H

cijkxijk,

∑(j,k)∈J×K

xijk = 1 ∀i ∈ I, (63)

∑(i,k)∈I×K

xijk = 1 ∀j ∈ J, (64)

∑(i,j)∈I×J

xijk = 1 ∀k ∈ K, (65)

xijk ∈ {0, 1} ∀(i, j, k) ∈ H. (66)

A number of applications of 3AP can be found in the literature (see Piersjalla [18,19]). 3AP isknown to be NP-hard [26]. As before, the set of feasible solutions to 3AP is noted as

F = {x ∈ RH | x satisfies (63)− (66)}

and we set P = conv(F).In [7], Balas and Saltzman study the polyhedral structure of P and introduce several classes

of facet-inducing inequalities. Let u, v ∈ H and define |u ∩ v| to be the numbers of coordinatesfor which the vectors u and v have the same value. Let C(u) = {w ∈ H | |u ∩ w| = 2} andC(u, v) = {w ∈ H | |u∩w| = 1, |w ∩ v| = 2}. We consider two classes of facet-inducing inequalitiesQ1(u) and P1(u, v) for P,

xu +∑

w∈C(u)

xw ≤ 1 ∀u ∈ H, (67)

xu +∑

w∈C(u,v)

xw ≤ 1 ∀u, v ∈ H, |u ∩ v| = 0. (68)

41

Page 42: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

0

0

3

2

1

0

1 2 3

1

2

3

(f) P1((0, 0, 3), (1, 3, 1))

(3, 0, 0)(0, 3, 1)(1, 1, 2)(3, 2, 3)

(2, 2, 0)(0, 3, 1)(1, 1, 2)(0, 0, 3)

(2, 1, 0)(1, 0, 1)(2, 3, 2)(3, 2, 3)

(0, 0, 3) 1/3 (0, 3, 1) 2/3(1, 0, 1) 1/3 (1, 1, 2) 2/3(2, 1, 0) 1/3 (2, 2, 0) 1/3(2, 3, 2) 1/3 (3, 0, 0) 1/3(3, 2, 3) 2/3

∑w∈C(0,0,1) xw = 1 1/3 > 1

∑w∈C((0,0,3),(1,3,1)) xw = 1 1/3 > 1

(a) x (b) λ1 = 13

(c) λ2 = 13

(d) λ3 = 13

(e) Q1(0, 0, 1)

Figure 21: Example of a decomposition into assignments.

Note that these include all the clique facets of the intersection graph of Kn,n,n [7]. In [6], Balasand Qi describe algorithms that solve the separation problem for the polyhedra defined by theinequalities Q1(u) and P1(u, v) in O(n3) time.

Balas and Saltzman consider the use of the classical Assignment Problem (AP) as a relaxationof 3AP in an early implementation of branch, relax, and cut [8]. The feasible region of the AP is

F ′ = {x ∈ RH | x satisfies (64)− (66)}.

The AP can be solved in O(n5/2 log(nC)) time where C = maxw∈H cw, by the cost-scaling algorithm[2]. The polyhedron PO is here described by constraints (63), the constraints Q1(u) for all u ∈ H,and the constrains P1(u, v) for all u, v ∈ H. Consider generation of a constraint of the form Q1(u)for some u ∈ H violated by a given s ∈ F ′. Let L(s) be the set of n triplets corresponding tothe nonzero components of s (the assignment from J to K). It is easy to see that if there existu, v ∈ L(s) such that u = (i0, j0, k0) and v = (i0, j1, k1), i.e., the assignment overcovers the set I,then both Q(i0, j0, k1) and Q(i0, j1, k0) are violated by s. Figure 21 shows the decomposition ofa vector x (a) the could arise during the execution of either the price and cut or decompose andalgorithms, along with a decomposition of x into a convex combination of assignments (b-d). Thepair of triplets (0, 3, 1) and (0, 0, 3) satisfies the condition just discussed and identifies two validinequalities, Q1(0, 3, 3) and Q1(0, 0, 1), that are violated by the second assignment, shown in (c).The latter also violates x and is illustrated in (e). This separation routine runs in O(n) time.

Now consider generation of a constraint of the form P1(u, v) for some u, v ∈ H violated bys ∈ F ′. As above, for any pair of assignments that correspond to nonzero components of sand have the form (i0, j0, k0), (i0, j1, k1), we know s violates P1((i0, j0, k0), (i, j1, k1)), ∀i 6= i0 and

42

Page 43: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

P1((i0, j1, k1), (i, j0, k0)),∀i 6= i0. The inequality P1((0, 0, 3), (1, 3, 1)) is violated by the second as-signment, shown in Figure 21(c). This inequality is also violated by x and is illustrated in (f).Once again, this separation routine runs in O(n) time.

8.3 Steiner Tree Problem

Let G = (V,E) be a complete undirected graph with vertex set V = {1, ..., |V |}, edge set E anda positive weight ce associated with each edge e ∈ E. Let T ⊂ V define the set of terminals.The Steiner Tree Problem (STP), which is NP-hard, is that of finding a subgraph that spans T(called a Steiner tree) and has minimum edge cost. In [13], Beasley formulated the STP as a sideconstrained Minimum Spanning Tree Problem (MSTP) as follows. Let r ∈ T be some terminal anddefine an artificial vertex 0. Now, construct the augmented graph G = (V , E) where V = V ∪ {0}and E = E ∪ {{i, 0} | i ∈ (V \ T ) ∪ {r}}. Let ci0 = 0 for all i ∈ (V \ T ) ∪ {r}. Then, the STPis equivalent to finding a minimum spanning tree (MST) in G subject to the additional restrictionthat any vertex i ∈ (V \ T ) connected by edge {i, 0} ∈ E must have degree one.

By associating a binary variable xe with each edge e ∈ E, indicating whether or not the edge isselected, we can then formulate the STP as the following integer program:

min∑e∈E

cexe,

x(E) = |V | − 1, (69)

x(E(S)) ≤ |S| − 1 ∀S ⊆ V , (70)

xi0 + xe ≤ 1 ∀e ∈ δ(i), i ∈ (V \ T ), (71)

xe ∈ {0, 1} ∀e ∈ E. (72)

Inequalities (69) and (70) ensure that the solution forms a spanning tree on G. Inequalities (70)are subtour elimination constraints (similar to those used in the TSP). Inequalities (71) are theside constraints that ensure the solution can be converted to a Steiner tree by dropping the edgesin E \ E.

The set of feasible solutions to the STP is

F = {x ∈ RE | x satisfies (69)− (72)}.

We set P = conv(F) as before. We consider two classes of valid inequalities for P that are liftedversions of the subtour elimination constraints (SEC).

x(E(S)) + x(E(S \ T | {0})) ≤ |S| − 1 ∀S ⊆ V, S ∩ T 6= ∅, (73)

x(E(S)) + x(E(S \ {v} | {0})) ≤ |S| − 1 ∀S ⊆ V, S ∩ T = ∅, v ∈ S. (74)

The class of valid inequalities (73) were independently introduced by Goemans [27], Lucena [43]and Margot, Prodon, and Liebling [47], for another extended formulation of STP. The inequalities(74) were introduced in [27, 47]. The separation problem for inequalities (73) and (74) can besolved in O(|V |4) time through a series of max-flow computations.

43

Page 44: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

11 1

22 2

33 3

0.5

444 5

0.5

55

66

6

6

6

0.5

6

0.5

(a) x (b) λ = 12 (c) λ = 1

2

Figure 22: Example of a decomposition into minimum spanning trees.

In [44], Lucena considers the use of MSTP as a relaxation of STP in the context of a branch,relax, and cut algorithm. The feasible region of the MSTP is

F ′ = {x ∈ RE | x satisfies (69), (70), (72)}.

The MSTP can be solved in O(|E| log |V |) time using Prim’s algorithm [55]. The polyhedron PO isdescribed by the constraints (71), (73), and (74). Constraints (71) can be represented explicitly, butwe must dynamically generate constraints (73) and (74). In order to identify an inequality of theform (73) or (74) violated by a given s ∈ F ′, we remove the artificial vertex 0 and find the connectedcomponents on the resulting subgraph. Any component of size greater than 1 that does not containr and does contain a terminal, defines a violated SEC (73). In addition, if the component does notcontain any terminals, then each vertex in the component that was not connected to the artificialvertex defines a violated SEC (74).

Figure 22 gives an example of a vector (a) that might have arisen during execution of eitherthe price and cut or decompose and cut algorithms, along with a decomposition into a convexcombination of two MSTs (b,c). In this figure, the artificial vertex is black, the terminals are grayand r = 3. By removing the artificial vertex, we easily find a violated SEC in the second spanningtree (c) with S equal to the marked component. This inequality is also violated by the optimalfractional solution, since x(E(S)) + x(E(S \ T | {0})) = 3.5 > 3 = |S| − 1. It should also be notedthat the first spanning tree (b), in this case, is in fact feasible for the original problem.

9 Conclusions and Future Work

In this paper, we presented a framework for integrating dynamic cut generation (outer methods)and traditional decomposition methods (inner methods) to yield new integrated methods that mayproduce bounds that are improved over those yielded by either technique alone. We showed therelationships between the various methods and how they can be viewed in terms of polyhedralintersection. We have also introduced the concept of structured separation and a related paradigmfor the generation of improving inequalities based on decomposition and the separation of solutions

44

Page 45: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

to a relaxation. The next step in this research is to complete a computational study using thesoftware framework introduced in Section 7 that will allow practitioners to make intelligent choicesbetween the many possible variants we have discussed.

References

[1] K. Aardal and S. van Hoesel. Polyhedral techniques in combinatorial optimization. StatisticaNeerlandica, 50:3–26, 1996.

[2] R. Ahuja, T. Magnanti, and J. Orlin. Network Flows: Theory, Algorithms, and Applications.Prentice Hall, Englewood Cliffs, NJ, 1993.

[3] D. Applegate, R. Bixby, V. Chvatal, and W. Cook. TSP cuts which do not conform to thetemplate paradigm. In Computational Combinatorial Optimization, pages 261–303. Springer,2001.

[4] D. Applegate, R. Bixby, V. Chvatal, and W. Cook. Implementing the Dantzig-Fulkerson-Johnson algorithm for large traveling salesman problems. Mathematical Programming, 97:91–153, 2003.

[5] P. Augerat, J. M. Belenguer, E. Benavent, A. Corberan, D. Naddef, and G. Rinaldi. Compu-tational Results with a Branch and Cut Code for the Capacitated Vehicle Routing Problem.Technical Report 949-M, Universite Joseph Fourier, Grenoble, France, 1995.

[6] E. Balas and L. Qi. Linear-time separation algorithms for the three-index assignment polytope.Discrete Applied Mathematics, 43:1–12, 1993.

[7] E. Balas and M.J. Saltzman. Facets of the three-index assignment polytope. Discrete AppliedMathematics, 23:201–229, 1989.

[8] E. Balas and M.J. Saltzman. An algorithm for the three-index assignment problem. OperationsResearch, 39:150–161, 1991.

[9] F. Barahona and R. Anbil. The volume algorithm: Producing primal solutions with a subgra-dient method. Mathematical Programming, 87:385–399, 2000.

[10] F. Barahona and D. Jensen. Plant location with minimum inventory. Mathematical Program-ming, 83:101–111, 1998.

[11] C. Barnhart, C. A. Hane, and P. H. Vance. Using branch-and-price-and-cut to solve origin-destination integer multi-commodity flow problems. Operations Research, 48:318–326, 2000.

[12] C. Barnhart, E. L. Johnson, G. L. Nemhauser, M. W. P. Savelsbergh, and P. H. Vance. Branchand price: Column generation for solving huge integer programs. Operations Research, 46:316–329, 1998.

[13] J.E. Beasley. A SST-based algorithm for the steiner problem in graphs. Networks, 19:1–16,1989.

45

Page 46: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

[14] J.E. Beasley. Lagrangean relaxation. In C.R. Reeves, editor, Modern Heuristic Techniques forCombinatorial Optimization. John Wiley & Sons, New York, 1993.

[15] A. Belloni and A. Lucena. A Lagrangian heuristic for the linear ordering problem. In M.G.C.Resende and J. Pinho de Sousa, editors, Metaheuristics: Computer Decision-Making, pages123–151. Kluwer Academic, 2003.

[16] F. C. Calheiros, A. Lucena, and C. de Souza. Optimal rectangular partitions. Networks,41:51–67, 2003.

[17] A. Caprara, M. Fischetti, and P. Toth. Algorithms for the set covering problem. Annals ofOperations Research, 98:353–371, 2000.

[18] P. Carraresi, A. Frangioni, and M. Nonato. Applying bundle methods to optimization of poly-hedral functions: An applications-oriented development. Technical Report TR-96-17, Univer-sita di Pisa, 1996.

[19] G. B. Dantzig and P. Wolfe. Decomposition principle for linear programs. Operations Research,8(1):101–111, 1960.

[20] G.B. Dantzig and R.H. Ramser. The truck dispatching problem. Management Science, 6:80–91,1959.

[21] M. Poggi de Aragao and E. Uchoa. Integer program reformulation for robust branch-and-cut-and-price. Working paper, Pontifıca Universidade Catolica do Rio de Janeiro, 2004. Availablefrom http://www.inf.puc-rio.br/˜uchoa/doc/rbcp-a.pdf.

[22] M.L. Fisher. The Lagrangian relaxation method for solving integer programming problems.Management Science, 27:1–18, 1981.

[23] M.L. Fisher. Optimal solution of vehicle routing problems using minimum k-trees. OperationsResearch, 42:626–642, 1994.

[24] R. Fukasawa, M. Poggi de Aragao, M. Reis, and E. Uchoa. Robust branch-and-cut-and-price forthe capacitated minimum spanning tree problem. In Proceedings of the International NetworkOptimization Conference, pages 231–236, Evry, France, 2003.

[25] M. Galati. DECOMP user’s manual. Technical report, Lehigh University Industrial andSystems Engineering, 2005.

[26] M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the Theory ofNP-Completeness. W. H. Freeman and Company, New York, 1979.

[27] M.X. Goemans. The steiner tree polytope and related polyhedra. Mathematical Programming,63:157–182, 1994.

[28] J. L. Goffin and J. P. Vial. Convex nondifferentiable optimization: A survey focused on theanalytic center cutting plane method. Technical Report 99.02, Logilab, Geneva, Switzerland,1999.

46

Page 47: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

[29] J. Gondzio and R. Sarkissian. Column generation with a primal-dual method. Technicalreport, University of Geneva, Logilab, HEC Geneva, 1996.

[30] M. Grotschel, L. Lovasz, and A. Schrijver. The ellipsoid method and its consequences incombinatorial optimization. Combinatorica, 1:169–197, 1981.

[31] M. Grotschel and M. Padberg. On the symmetric travelling salesman problem I: Inequalities.Mathematical Programming, 16:265–280, 1979.

[32] M. Grotschel and M. Padberg. On the symmetric travelling salesman problem II: Liftingtheorems and facets. Mathematical Programming, 16:281–302, 1979.

[33] M. Guignard. Efficient cuts in Lagrangean “relax-and-cut” schemes. European Journal ofOperational Research, 105:216–223, 1998.

[34] M. Guignard. Lagrangean relaxation. Top, 11:151–228, 2003.

[35] M. Held and R. M. Karp. The traveling salesman problem and minimum spanning trees.Operations Research, 18:1138–1162, 1970.

[36] D. Huisman, R. Jans, M. Peeters, and A. Wagelmans. Combining column generation andLagrangian relaxation. Technical report, Erasmus Research Institute of Management, Rotter-damn, The Netherlands, 2003.

[37] M. Hunting, U. Faigle, and W. Kern. A Lagrangian relaxation approach to the edge-weightedclique problem. European Journal of Operational Research, 131:119–131, 2001.

[38] N. Kohl, J. Desrosiers, O.B.G. Madsen, M.M. Solomon, and F. Soumis. 2-path cuts for thevehicle routing problem with time windows. Transportation Science, 33:101–116, 1999.

[39] L. Kopman. A New Generic Separation Routine and Its Application In a Branch and CutAlgorithm for the Capacitated Vehicle Routing Problem. PhD thesis, Cornell University, May1999.

[40] G. Laporte, Y. Nobert, and M. Desrouchers. Optimal routing with capacity and distancerestrictions. Operations Research, 33:1050–1073, 1985.

[41] J. Linderoth and M. Galati. Knapsack constrained circuit problem. Unpublished workingpaper, 2004.

[42] M. E. Lubbecke and J. Desrosiers. Selected topics in column generation. Technical Report008-2004, Technische Universitat Berlin, 2004.

[43] A. Lucena. Tight bounds for the steiner problem in graphs. Talk given at TIMS-XXX-SOBRAPO XXIII Joint International Meeting, Rio de Janeiro, July 1991.

[44] A. Lucena. Steiner problem in graphs: Lagrangean relaxation and cutting planes. COALBulletin, 28:2–8, 1992.

[45] A. Lucena. Non delayed relax-and-cut algorithms. Annals of Operations Research, 140(1):375–410, 2005.

47

Page 48: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

[46] J. Lysgaard, A.N. Letchford, and R.W. Eglese. A new branch-and-cut algorithm for thecapacitated vehicle routing problem. Mathematical Programming, 100:423–445, 2004.

[47] F. Margot, A. Prodon, and T.M. Liebling. Tree polyhedron on 2-tree. Mathematical Program-ming, 63:183–192, 1994.

[48] C. Martinhon, A. Lucena, and N. Maculan. Stronger k-tree relaxations for the vehicle routingproblem. European Journal of Operational Research, 158:56–71, 2004.

[49] D. Miller. A matching based exact algorithm for capacitated vehicle routing problems. ORSAJournal on Computing, 7:1–9, 1995.

[50] M. Muller-Hannemann and A. Schwartz. Implementing weighted b-matching algorithms: To-wards a flexible software design. In Proceedings of the Workshop on Algorithm Engineeringand Experimentation (ALENEX99), volume 1619 of Lecture notes in computer science, pages18–36, Baltimore, MD, 1999. Springer-Verlag.

[51] D. Naddef and G. Rinaldi. Branch-and-cut algorithms for the capacitated VRP. In P. Tothand D. Vigo, editors, The Vehicle Routing Problem, pages 53–84. SIAM, 2002.

[52] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization. Wiley, New York,1988.

[53] M. Padberg and G. Rinaldi. An efficient algorithm for the minimum capacity cut problem.Mathematical Programming, 47:19–36, 1990.

[54] A. Pigatti. Modelos e algoritmos para o problema de alocao generalizada e aplicacoes. PhDthesis, Pontifıcia Universidade Catolica do Rio de Janeiro, 2003.

[55] R. Prim. Shortest connection networks and some generalizations. Bell System TechnicalJournal, 36:1389–1401, 1957.

[56] T K Ralphs. Parallel Branch and Cut for Vehicle Routing. PhD thesis, Cornell University,may 1995.

[57] T K Ralphs, L Kopman, W R Pulleyblank, and L E Trotter Jr. On the capacitated vehiclerouting problem. Mathematical Programming, 94:343–359, 2003.

[58] L. M. Rousseau, M. Gendreau, and D. Feillet. Interior point stabilization for column generation.Working paper, Cahier du Gerad, 2003. Available from http://www.lia.univ-avignon.fr/

fich_art/380-IPS.pdf.

[59] V. Chvatal. Linear Programming. W.H. Freeman and Company, 1983.

[60] J.M. van den Akker, C.A.J. Hurkens, and M.W.P. Savelsbergh. Time-indexed formulationsfor machine scheduling problems: Column generation. INFORMS Journal on Computing,12:111–124, 2000.

[61] F. Vanderbeck. Lot-sizing with start-up times. Management Science, 44:1409–1425, 1998.

48

Page 49: Decomposition in Integer Linear Programmingted/files/papers/DECOMP04.pdfDecomposition in Integer Linear Programming M.V. Galati 1 and Ted K. Ralphsy2 1Advanced Analytics - Operations

[62] G. Wei and G. Yu. An improved O(n2 log n) algorithm for the degree-constrained minimumk-tree problem. Technical report, The University of Texas at Austin, Center for Managementof Operations and Logistics, 1995.

49