Quality of LP-based Approximations for Highly Combinatorial Problems

Lucian Leahu and Carla Gomes

Computer Science DepartmentCornell University

Motivation Increasing interest in combining Constraint Satisfaction Problem

(CSP) formulations and Linear Programming (LP) based techniques for solving hard computational problems.

Successful results for solving problems that are a mixture of linear constraints – where LP excels – and combinatorial constraints – where CSP excels.

In a purely combinatorial setting,

surprisingly difficult to effectively

integrate LP- and CSP-based techniques

Goal

Study and characterize the quality of LP based heuristics

for highly combinatorial problems.

Research Questions Is the quality of LP-based Approximations related to

the structure of the problem? (Typical case, rather than worst case)

How is the quality of LP-based Approximations influenced by different formulations of the problem?

Does the LP relaxation provide a global perspective of the search space? Is the LP relaxation good as a heuristic to guide complete solvers?

Outline A highly combinatorial search problem ---

quasigroup completion problem (QCP) LP-based formulations for QCP

Assignment based formulation Packing formulation

Quality of LP based approximations LP as a global search heuristic Conclusions

Latin Squares or Quasigroups Given an N X N matrix, and given N colors, a

quasigroup of order N is a a colored matrix, such that:

all cells are colored. each color occurs exactly once in each row. each color occurs exactly once in each column.

Quasigroup or Latin Square(Order 4)

Latin Squares/Quasigroups Completion Problem Given a partial assignment of colors (10 colors in this

case), can the partial Latin Square be completed so we obtain a full square?

Latin Squares/Quasigroups Completion Problem Given a partial assignment of colors (10 colors in this

case), can the partial Latin Square be completed so we obtain a full square?

Example:

Structure of this problem characterizes several real-world applications: e.g., Timetabling, sports scheduling, rostering, routing, etc.

Quasigroup with Holes (QWH)

Given a full quasigroup, “punch” holes into it

QWH is NP-Hard.

32% holes

Advantage: we know the optimal value.

LP-based formulations for QCP

Assignment Formulation

Variables -

}1,0{ijkx

....,,2,1,,;, nkjikcolorhasjicellijkx

3n

kijPLStskjikjix ..,,1,,

Max number of colored cells

s.t. at most one color per cell:

a color appears at most once per row

a color appears at most once per column

Assignment Formulation

New Phase Transition Phenomenon:Integrality of LP

Note: standard phase transition curves are w.r.t existence of solution)

Sudden phase Transition in solution integrality of LP relaxation

and it coincides with the hardest area

holes/n^1.55

No

of b

ackt

rack

sM

ax v

alue

of L

P R

elax

atio

n

Packing formulation

Max number of colored cells in the selected patterns

s.t. one pattern per family

a cell is covered at most by one pattern

Families of patterns

(partial patterns are not shown)

Packing formulation

Previous Results 0.5 approximation based on Assignment formulation –

Kumar et al. – 1999

(1-1/e ≈ 0.63) approximation based on Packing formulation – Gomes, Regis, Shmoys – 2003

Use of LP to select variables and values and to prune search trees – Refalo et al. – 1999, 2000

No typical case results on the quality of LP based approximation

Quality of LP-based Approximations

Approximation Schemes

LP Formulations: Assignment formulation; Packing formulation;

Approximation scheme: solve the LP relaxation and interpret the resulting solution as a

probability distribution;

Order for Variable SettingUniformly at RandomGreedy RandomGreedy Deterministic

Increasing greediness

Uniformly at Random

Uniformly at Random

Uniformly at Random

holes/n^1.55 holes/n^1.55

% o

f col

ored

hol

es

% o

f col

ored

hol

es

Uniformly Random - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality stabilizes in the under constrained area

Random LP Packing does better, since the corresponding LP relaxation is stronger

Random LP Packing is a 1 – 1/e≈0.63 approximation, while LP assignment ½ approximation.

% o

f col

ored

hol

es

holes/n^1.55

Greedy Random

Greedy Random

Greedy Random%

of c

olor

ed h

oles

% o

f col

ored

hol

esholes/n^1.55 holes/n^1.55

Greedy Random - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality increases in the under constrained area --- info provided by LP is used in a more greedy way (more valuable); forward checking also improves quality.

Random LP Packing does slightly worse, since it optimizes an entire matching

% o

f col

ored

hol

es

holes/n^1.55

Greedy Deterministic

Greedy Deterministic

Greedy Deterministic - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality increases in the under constrained area --- info provided by LP is used in a more greedy way and deterministically (more valuable); forward checking also improves quality.

Random LP Packing does slightly worse, since it is less greedy (sets an entire matching), doesn’t use as much lookahead

% o

f col

ored

hol

es

holes/n^1.55

Comparison with Pure Random Strategy

% o

f col

ored

hol

es

holes/n^1.55

LP as a Global Search Heuristic

Can LP guide complete solvers?

Use an LP relaxation to set a certain percent of variables (the highest values)

Run a complete solver on the resulting instances and check if it is still completable (we start with a PLS that is completable)

LP as a Global Search Heuristic - Results

1 hole

% o

f sat

isfia

ble

inst

ance

s

holes/n^1.55

5%

holes/n^1.55

% o

f sat

isfia

ble

inst

ance

s

Conclusions

Quality of approximation is directly correlated with phase transition phenomenon – closely related to constrainedness regions of the problem (sharp decrease in the critical region)

New phase transition in the integrality of the LP relaxation solution

Typical case analysis – although theoretical bounds for LP packing are stronger, the empirical results for enhanced versions of approximations (with forward-checking) seem to indicate that LP approximations based on the assignment formulation are better (but difficult to analyze theoretically)

LP can provide useful high level guidance + should be combined with random restart strategies to recover from potential mistakes made at the top of the tree

Quality of LP-based Approximations for Highly Combinatorial Problems

Lucian Leahu and Carla Gomes

Computer Science DepartmentCornell University