Biogas Plant Control and Optimization Using Computational ...
Optimization on the Computational...
Transcript of Optimization on the Computational...
Optimization on the Computational Grid
Jeff Linderoth
ISE DepartmentCOR@L Lab
Lehigh [email protected]
Enterprise-Wide Optimization (EWO) Tele-SeminarAllentown, PAMay 9, 2006
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 1 / 67
An Abject Apology
Much of this work was done long ago
There is no original material!I even have reused many of the same hackneyed jokes
I have a good excuse
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 2 / 67
My Good Excuse—Jacob Linderoth
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 3 / 67
Outline
What I lack in quality, I make up for in quantity: ≥ 70 slides
What is The Computational Grid?
Tools for using the Computational Grid
Condor and MW
Solving large optimization problem instances on the ComputationalGrid
Two-stage stochastic linear programmingQuadratic Assignment Problem
Theme of the Talk
Fit your algorithm to your computational platform!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 4 / 67
The Grid Richard Dawson Rules!
Come on Let’s Play the Feud
‘‘100 People Surveyed. Top5 answers are on the board.Here’s the question...’’
Name one common use of the Internet
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 5 / 67
The Grid Richard Dawson Rules!
The Big Board
1 email
2 Looking up answers to homeworkproblems
3 Looking up stock quotes
4 Downloading MP3s
5 Looking at pictures of Anna Kournikova
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 6 / 67
The Grid Richard Dawson Rules!
Strike!
DoingComputations
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 7 / 67
The Grid Building a Grid
People envision a “Computational Grid” much like the national powergrid
Users can seamlessly draw computational power whenever they need itMany resources can be brought together to solve very large problemsGives application experts the ability to solve problems of unprecedentedscope and complexity, or to study problems which they otherwise wouldnot.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 8 / 67
The Grid Building a Grid
This ain’t easy!
User access and security
Who should be allowed to tap in?
Interfaces
How should they tap in?
Heterogeneity
Different hardware, operating systems, and software
Dynamic
Participating Grid resources may come and goFault-Tolerance is very important!
Communicationally challenged
Machines may be very far apart ⇒ slow communication.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 9 / 67
The Grid Building a Grid
Building a Grid
There have been lots of software tools that accomplish these goals tovarying degrees.
One problem remains: GREED!Most people don’t want to contribute “their” machine!
How to induce people to contribute their machine to the Grid?
Screensaver – seti@homeSocial Welfare – fightaids@homeOffer frequent flyer miles – company went bankruptLet the people keep control over their machineGive donaters a chance to use the Grid
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 10 / 67
The Grid Condor
Tool I – Condor
Peter KellerMiron LivnyErik PaulsenRajesh Raman
Marvin SolomonTodd Tannenbaum
Doug ThainDerek Wright
http://www.cs.wisc.edu/condor
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 11 / 67
The Grid Condor
What is Condor?
Manages collections of “distributively owned” workstations
User need not have an account or access to the machineWorkstation owner specifies conditions under which jobs are allowed torunAll jobs are scheduled and “fairly” allocated among the pool
How does it do this?
Scheduling/MatchmakingJobs can be checkpointed and migratedRemote system calls provide the originating machines environment
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 12 / 67
The Grid Condor
Matchmaking
MyType = JobTargetType = MachineOwner = ralphsCmd = cplexArgs = seymour.d10.mpsHasCplex = TRUEMemory ≥ 64Rank = KFlopsArch = SUN4uOpSys = SOLARIS27 ||
SOLARIS28
MyType = MachineTargetType = JobName = nova9HasCplex = TRUEArch = SUN4uOpSys = SOLARIS27Memory = 256KFlops = 53997RebootedDaily = TRUE
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 13 / 67
The Grid Condor
Checkpointing/Migration
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 14 / 67
The Grid Condor
Other Condor Features
Pecking Order
Users are assigned priorities based on the number of CPU cycles theyhave recently usedIf someone with higher priority wants a machine, your job will bebooted off
Flocking
Condor jobs can negotiate to run in other Condor pools.
Glide-in
Globus (Grid computing toolkit from Argonne) provides a “front-end”to many traditional supercomputing sites.Submit a Globus job which creates a temporary Condor pool on thesupercomputer, on which users jobs may run.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 15 / 67
The Grid Condor
Condor is Really Taking Off
Companies Presenting at Recent Condor Week 2006
JP Morgan
Altera Corporation
Hartford Life
Micron
UBS
Yahoo
http://www.cs.wisc.edu/condor/CondorWeek2006/
http://www.cs.wusc.edu/condor
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 16 / 67
The Grid MW
Grid computing applications
What Condor is good at...
Naturally/Pleasantly/Embarassingly Parallel Applications
The computation can a priori be broken into tasks
Monte Carlo Methods, Code Crackingseti@home – the search for intelligent life
What Condor is (not so) good at...
Parallel Applications with non-trivial control structures
Like Optimization Algorithms for EWO!
To make parallel algorithms dynamically adjustable and fault-tolerant,we could use the master-worker paradigm
What is the master-worker paradigm, you ask?
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 17 / 67
The Grid MW
Master-Worker!Fee
dM
e!OK!
Tutor
Me!
OK!
Master assigns tasks to theworkers
Workers perform tasks, andreport results back to master
Workers do not communicate(except through the master)
Simple!
Fault-tolerant
Dynamic
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 18 / 67
The Grid MW
Tool II – MW
Wen-Han Goh
Sanjeev KulkarniGreg ThainMike Yoder{
Jean-Pierre GouxJeff Linderoth
http://www.cs.wisc.edu/condor/mw
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 19 / 67
The Grid MW
MW
There are three abstraction is the master-worker paradigm: Master,Worker, and Task.
MW is a software package that encapsulates these abstractions
C++ abstract classesUser writes 10 functionsThe MWized code will transparently adapt to the dynamic andheterogeneous environment
MW also has abstract layer to resource management andcommunications packages
Condor/PVM, Condor/FilesCondor/Unix SocketsSingle processor
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 20 / 67
The Grid MW
MW Classes
MWMaster
get userinfo()setup initial tasks()pack worker init data()act on completed task()
MWTask
(un)pack work(un)pack result
MWWorker
unpack worker init data()execute task()
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 21 / 67
The Grid MW
MWApplications
MWFATCOP (Chen, Ferris, Linderoth) – A branch and cut code for linearinteger programming
MWMINLP (Goux, Leyffer, Nocedal) – A branch and bound code fornonlinear integer programming
MWSVM (Ferris, Voelker) – A column generation code for solving supportvector machine problems
MWQPBB (Linderoth) – A branch-and-bound code for nonconvex quadraticprogramming
MWAND (Linderoth, Shen) – A (preliminary) nested-decomposition code formultistage stochastic linear programming
MWLShaped/MWATR (Linderoth, Shapiro, Wright) – A (trust-region)cutting plane code for linear stochastic programming and verification ofsolution quality
MWQAP (Anstreicher, Brixius, Goux, Linderoth) – A branch and boundcode for solving the quadratic assignment problem
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 22 / 67
Stochastic Programming Background
Stochastic Programming Collaborators
Alex Shapiro
ISyEGeorgia Tech
Steve Wright
Computer Science DepartmentUniversity of Wisconsin-Madison
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 23 / 67
Stochastic Programming Background
Stochastic Programming
A Stochastic Program
minx∈X
{f(x)def= EP F (x, ω)}
F (x, ω) is a real-valued functionof two vector variables.
x ∈ Rn and ω ∈ Rd
ω is a random vector havingprobability distribution P
X ⊆ <n
A Two-Stage Stochastic LP
minx≥0,Ax=b
cT x +Q(x)
Q(x)def= EP [Q(x, ω)]
The recourse problem:
Q(x, ω)def= min qT y
Wy = h(ω)− T (ω)x
y ≥ 0
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 24 / 67
Stochastic Programming The LShaped Method
Best-Known Solution Procedure
METH O D
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 25 / 67
Stochastic Programming The LShaped Method
Two-Stage Stochastic Linear Programming
We assume that the P has finite support, so ω has a finite number ofpossible realizations (scenarios):
Q(x) =N∑
i=1
piQ(x, ωi)
For a partition of the N scenarios into chunks N1,N2, . . .Nt, letQ[j](x) be the contribution of the jth chunk to Q(x):
Q[j](x)def=
∑i∈Nj
piQ(x, ωi)
so then Q(x) =∑t
j=1Q[j]
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 26 / 67
Stochastic Programming The LShaped Method
Important (and well-known) Facts
Q(x, ωi), Q[·](x), and Q(x) are piecewise linear convex functions of x.
If πi is an optimal dual solution to the linear program correspondingto Q(x, ωi), then −T T
i πi ∈ ∂Q(x, ωi)
gj(x)def=
∑i∈Nj
−piTTi πi ∈ ∂Q[j](x).
Represent Q[j](x) by an artificial variable θj and find supportingplanes for θj
θj ≥ gj(xk)T x + (Q[j](x
k)− gTj xk) (∗)
Evaluation of Q(x) is separable
We can solve linear programs corresponding to each Q(x, ωi)independently – in parallel!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 27 / 67
Stochastic Programming The LShaped Method
Worth 1000 Words
x
Q(x)
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 28 / 67
Stochastic Programming The LShaped Method
Worth 1000 Words
x
Q(x)
xk
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 29 / 67
Stochastic Programming The LShaped Method
Worth 1000 Words
x
Q(x)
x1x2
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 30 / 67
Stochastic Programming The LShaped Method
Multicut L-shaped method
M
s1 s2 s3 s4 s3
M
s1 s2 s3 s4 s5
1 Solve the masterproblem M with thecurrent θj-approximationsto Q[j](x) for xk.
2 Solve the subproblems,(sj) evaluating Q[j](x
k)and obtaining asubgradient gj(x
k). Addinequalities (*) to themaster problem
3 k = k+1. Goto 1.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 31 / 67
Stochastic Programming The LShaped Method
A Trust Region Method
The LShaped method suffers from “stability” problems,
Especially in early iterations when a “good enough” model of Q(x) isnot knownEspecially bad if started from a good guess at the solution
Borrow the trust region concept from NLP
At iteration k, impose constraints ‖x− xk‖∞ ≤ ∆k
Accept new iterate if it improves the objective by a “sufficient”amount. Potentially increase ∆k.Otherwise, improve the estimation of Q(xk), resolve master problem,and potentially reduce ∆k.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 32 / 67
Stochastic Programming The LShaped Method
Stop Thief!
“Lesser artists borrow, great artists steal.”
– Igor Stravinsky
Stabilizing Bender’s decomposition is hardly a new idea
Marsten, Hogan, Blankenship (’75)Lemarechal (’75, . . . )Kiwiel (’83, . . . )Ruszczynski (’86)Neame, Boland, and Ralph (’98)
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 33 / 67
Stochastic Programming The LShaped Method
minor Differences
We like ‖ · ‖∞ trust region
The master problem is just an LP with modified boundsDegeneracy has not reared its ugly head
(Perhaps due to better LP solvers)
Convergence analysis is somewhat different
Convergence for arbitrary subgradients – not necessarily extreme pointdual solutions π(ωi).
Liberal cut deletion from master is possible
This is especially important in a grid computing implementation
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 34 / 67
Stochastic Programming Gridifying the LShaped Method
Preparing for the Grid
Work
One or more scenario chunks Nj1 , . . .NjCand point (x)
Result
A subgradient of each of the Q[jk](x).
act on completed taskAdd subgradient inequalities to master problemSolve master problem if all workers have reported their results for theiteration
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 35 / 67
Stochastic Programming Gridifying the LShaped Method
Headaches!
Solving the master problem is a“synchronization point” of thealgorithm
Amdahl’s Law: Parallel efficiency islimited by the amount ofsynchronization.
This synchronization problem is MUCH worse in Computational Gridcomputing environments!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 36 / 67
Stochastic Programming Gridifying the LShaped Method
Worker Usage—Number of Idle Workers
0
10
20
30
40
50
60
70
80
90
100
0 500 1000 1500 2000 2500 3000 3500 4000
Num
ber
of Id
le W
orke
rs
t (sec.)Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 37 / 67
Stochastic Programming Gridifying the LShaped Method
Stamp Out Synchronicity!
On a Grid, different processors act at different speeds,
Many may wait idle for the “slowpoke”
Even worse, grid computing tools can fail to inform the user that theirworker has failed!
Asynchronicity is key!
Asynchronous methods are preferred for traditional parallel computingenvironments. They are nearly required for Grid Computing environ-ments!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 38 / 67
Stochastic Programming Gridifying the LShaped Method
ATR – An Asynchronous Trust Region Method
Keep a “basket” B of trial points for which we are evaluating theobjective function
Make decision on whether or accept new iterate xk+1 after entireQ(xk) is computed
Populate the basket quickly by initially solving the master problemafter only α% of the scenario LPs have been solved
Greatly reduces the synchronicity requirements
Might be doing some “unnecessary” work – the candidate pointsmight be better if you waited for complete information from thepreceding iterations
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 39 / 67
Stochastic Programming Solving Big Instances
The World’s Largest LP
Storm – A stochastic cargo-flight scheduling problem (Mulvey andRuszczynski)
We aim to solve an instance with 10,000,000 scenarios
x ∈ <121, yk ∈ <1259
The deterministic equivalent LP is of size
A ∈ <985,032,889×12,590,000,121
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 40 / 67
Stochastic Programming Solving Big Instances
The Super Storm Computer
Number Type Location
184 Intel/Linux Argonne
254 Intel/Linux New Mexico
36 Intel/Linux NCSA
265 Intel/Linux Wisconsin88 Intel/Solaris Wisconsin239 Sun/Solaris Wisconsin
124 Intel/Linux Georgia Tech90 Intel/Solaris Georgia Tech13 Sun/Solaris Georgia Tech
9 Intel/Linux Columbia U.10 Sun/Solaris Columbia U.
33 Intel/Linux Italy (INFN)
1345
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 41 / 67
Stochastic Programming Solving Big Instances
TA-DA!!!!!
Statistics from Storm Computation
Wall clock time 31:53:37CPU time 1.03 Years
Avg. # machines 433Max # machines 556Parallel Efficiency 67%
Master iterations 199CPU Time solving the master problem 1:54:37
Maximum number of rows in master problem 39647
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 42 / 67
Stochastic Programming Solving Big Instances
Number of Workers
0
100
200
300
400
500
600
0 20000 40000 60000 80000 100000 120000 140000
#wor
kers
Sec.Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 43 / 67
Quadratic Assignment Problem Background
The Quadratic Assignment Problem
Mathematical Formulation
minπ∈Π
n∑i=1
n∑j=1
aijbπ(i)π(j) +n∑
i=1
ciπ(i)
Assign facilities to locations
QAP is NP-”Super”-Hard
Branch and Bound is themethod of choice, but very fewtight, computable, bounds exist.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 44 / 67
Quadratic Assignment Problem Background
QAP Collaborators
{Kurt Anstreicher
University of Iowa
{Nate Brixius
Micro$oft
{Jean-Pierre Goux
Argonne, Northwestern, and Artelys
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 45 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
How to solve BIG problems with Branch and Bound
1 Start with a state of the art bounding technique2 Do intelligent search and branching
We employ/extend “strong branching” in this non-linear contextUse different branching strategies depending on the relative difficulty ofa node
3 Pick a very powerful computing platform
The Computational Grid!
4 Fit the algorithm to the platform
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 46 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Branch and Bound
Kurt Anstreicher and Nate Brixius showed that the solution to thefollowing problem provides a lower bound to the solution of QAP:
min f(X) ≡ vec(X)T Q vecX + C •X
such that Xe = XT e = e, X ≥ 0.
Q ≡ (B ⊗A)− (I ⊗ S)− (T ⊗ I)
S and T are obtained from the spectral decompositions of A and B
There are more details that I don’t understand
This is a convex quadratic programming problem relaxation
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 47 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
An Oldie but a Goodie
The Frank-Wolfe algorithm is used to solve the quadraticprogramming subproblems
1. X∗k = arg minX∈Π{∇f(Xk) •X}. Linear Assignment Problem
2. Xk+1 = Xk + α(X∗k −Xk), where 0 ≤ α ≤ 1 minimizes f(Xk+1).
Closed form solution
3. Go to 1.
Each X∗k comes with a matrix of “reduced costs”, from which a valid
lower bound can be deduced: zOPT ≥ f(Xk)− Uk •Xk.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 48 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
How to solve BIG problems with Branch and Bound
1 Start with a state of the art bounding technique2 Do intelligent search and branching
We employ/extend “strong branching” in this non-linear contextUse different branching strategies depending on the relative difficulty ofa node
3 Pick a very powerful computing platform
The Computational Grid!
4 Fit the algorithm to the platform
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 49 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Branch and Bound
We do not do “normal” IP branching
Instead do n-ary branching
Fix each assignment in arow or column
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �
� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �
� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �
� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �
� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �
� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �
� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �� � � � �
� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �
� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �� � � � � � � � � �
� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �
� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �� � �
! !! !! !! !! !! !! !! !! !! !! !! !! !
" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "" " " " "
# # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # ## # # # #
$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $
% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % %% % % % % % % % % Facility 3
Location 2
Also adapted strong-branching to this nonlinear context.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 50 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Strong Branching
There are many “unsatisfied entities”, how do we decide on which tobranch?
We would like to select an entity that will increase the bound as muchas possible
MIP – Tentatively select a fractional variable, and perform a smallnumber of dual simplex pivots
QAP – Tentatively select a row or column and perform a smallnumber of Frank-Wolfe iterations
Branch on entity where the lower bounds improve “the most”
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 51 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
How to solve BIG problems with Branch and Bound
1 Start with a state of the art bounding technique2 Do intelligent search and branching
We employ/extend “strong branching” in this non-linear contextUse different branching strategies depending on the relative difficulty ofa node
3 Pick a very powerful computing platform
The Computational Grid!
4 Fit the algorithm to the platform
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 52 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Our Computational Grid
Number Type Location414 Intel/Linux Argonne96 SGI/Irix Argonne
1024 SGI/Irix NCSA16 Intel/Linux NCSA45 SGI/Irix NCSA246 Intel/Linux Wisconsin146 Intel/Solaris Wisconsin133 Sun/Solaris Wisconsin190 Intel/Linux Georgia Tech94 Intel/Solaris Georgia Tech54 Intel/Linux Italy (INFN)25 Intel/Linux New Mexico5 Intel/Linux Columbia U.10 Sun/Solaris Columbia U.12 Sun/Solaris Northwestern
2510
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 53 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
How to solve BIG problems with Branch and Bound
1 Start with a state of the art bounding technique2 Do intelligent search and branching
We employ/extend “strong branching” in this non-linear contextUse different branching strategies depending on the relative difficulty ofa node
3 Pick a very powerful computing platform
The Computational Grid!
4 Fit the algorithm to the platform
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 54 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
MW Implementation
Fitting the B & B algorithm into the master-worker paradigm is notground-breaking research
We must avoid “contention” at the master
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 55 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
All The Queueing Theory I Know
We can reduce contention in two ways
1. Increase the service rate2. Reduce the arrival rate
A parallel depth-first oriented strategy achieves these goals.
Available worker is given “deepest” node by masterWorker examines the subtree rooted at this node in a depth-firstfashion for t seconds.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 56 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Parallel Depth-First Search
����
����
Master Task Pool
Worker A
sent back to masterUnexplored nodes
Worker B
Worker C
Other “standard” searchstrategies fail completely!
Too much memoryrequired at masterToo many nodes passedback to master
Don’t try this at home!
If you don’t have a goodupper bound with which tofathom, this can failmiserably!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 57 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Truth in Advertising
The parallel depth-first search strategy is awful too!
Define the parallel efficiency:
η =Σ(Time workers spend executing tasks)
Σ(Time workers are available)
In our initial implementation, η = 0.41
Since there is very little synchronization required in the algorithm,this number is shockingly low!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 58 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Deducing the Problem
We may want the workers toexamine a subtree for tseconds, but that doesn’tmean that there are tseconds of work!
A histogram of task times:
0
500
1000
1500
2000
2500
3000
0.1 1
� �
10
� �
100
� �
�
�
�
� �
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 59 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Elementary, Dear Watson
Make sure that workers onlypass back nodes that willhave enough “meat”
Order children so that“easy” ones are first in theDFS stackAllow additional time forworkers to pop up the DFSstack, finishing offremaining easy nodes.
η improved to 0.9
0
150
300
450
600
0.1 1�� �
10�
100�
300
� �
500
� �
���� �
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 60 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
The Holy Grail
(NUG30) (n = 30) has been the “holy-grail” of computational QAPresearch for > 30 years
In 2000, Anstreicher, Brixius, Goux, & Linderoth set out to solve thisproblem
Using an old idea of Knuth, we estimated the CPU time required to solveNUG30 to be 5-10 years on a fast workstation
We’d better get a pretty power computing platform, like the ComputationalGrid.
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 61 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
NUG30 is solved!
14, 5, 28, 24, 1, 3, 16, 15, 10, 9, 21, 2, 4, 29, 25, 22, 13, 26, 17, 30, 6, 20, 19, 8, 18, 7, 27, 12, 11, 23
“My father used 3.46× 108 CPU seconds, and all I got wasthis lousy permutation”
Wall Clock Time: 6:22:04:31Avg. # Machines: 653
CPU Time: ≈ 11 yearsNodes: 11,892,208,412LAPs: 574,254,156,532
Parallel Efficiency: 92%
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 62 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Workers
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 63 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
KLAPS
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 64 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Even More Wasted CPU Time
KRA30B KRA32 THO30
Wall Clock Time (Days) 3.79 12.3 17.2Avg. # Machines 462 576 661Max. # Machines 780 1079 1307CPU Time (Years) 4.32 15.2 24.7
Nodes 5.14× 109 16.7× 109 34.3× 109
LAPs 188× 109 681× 109 1.13× 1012
Parallel Efficiency: 92% 87% 89%
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 65 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
Conclusions
The Computational Grid is a powerful computing platform
With the Master-Worker paradigm, it is relatively easy to implementand run on the Grid.
With a little tuning, many algorithms can be “shoehorned” to runeffectively with the Master-Worker paradigm, and hence on the Grid.
You have to fit your algorithm to the computational platform!
The boost in computing power that you get by making yourcode/algorithm Grid-enabled is worth the effort!
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 66 / 67
Quadratic Assignment Problem Practical Branch-and-Bound
The End!
We want YOU to use the Computational Grid and MW.
Papers at http://www.optimization-online.org
http://www.cs.wisc.edu/condor/mw
mailto:[email protected]
Jeff Linderoth (Lehigh University) Optimization on the Computational Grid EWO Tele-Seminar 67 / 67