Innovations in CPLEX performance and solver capabilities
-
Upload
ibm-decision-optimization -
Category
Software
-
view
1.524 -
download
7
description
Transcript of Innovations in CPLEX performance and solver capabilities
Decision Optimization
INFORMS 2013 Workshop - Innovations
INFORMS Fall Conference 2013
Building a Smarter Planet withDecision Optimization
© 2013 IBM Corporation
Decision Optimization
2 INFORMS 2013 Workshop - Innovations
Workshop presentations
3:00 - 3:30: Innovations in CPLEX performance and solver capabilities –Mary Fenelon
3:30 – 4:15: Expert tips and tricks for the OR practitioner: Everything you need to know about CPLEX-powered modeling and solving – Ed Klotz
4:30 - 5:10 From "Good Modeling" to developing and deploying great enterprise applications – Yianni Gamvros, Yana Ageeva, John Chaves
5:10 – 5:30 CPLEX-based tools and advanced analytics for Academia –Dennis Bly
Decision Optimization
INFORMS 2013 Workshop - Innovations
INFORMS Fall Conference 2013
Innovations in CPLEX performance and solver capabilities
Mary Fenelon
Development Manager
IBM ILOG CPLEX Optimization Studio
© 2013 IBM Corporation
Decision Optimization
4 INFORMS 2013 Workshop - Innovations
Disclaimer
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM®benchmarks in a controlled environment. The actual throughput orperformance that any user will experience will vary depending upon manyfactors, including considerations such as the amount of multiprogramming inthe user's job stream, the I/O configuration, the storage configuration, and theworkload processed. Therefore, no assurance can be given that an individualuser will achieve results similar to those stated here.
© 2013 IBM Corporation
Decision Optimization
5 INFORMS 2013 Workshop - Innovations
Agenda
New features in ILOG CPLEX Optimization Studio
Distributed computing options for CPLEX Optimizer
Performance
© 2013 IBM Corporation
Decision Optimization
6 INFORMS 2013 Workshop - Innovations
Agenda
New features in ILOG CPLEX Optimization Studio
Distributed computing options for CPLEX Optimizer
Performance
© 2013 IBM Corporation
Decision Optimization
7 INFORMS 2013 Workshop - Innovations
Model development Tools
Optimization Engines
Math Programming
CPLEX Optimizers
(Simplex, Barrier, Mixed
Integer)
Constraint Programming
Constraint-based scheduling
CPLEX CP Optimizers
CPLEX Studio (IDE) - OPL Modeling Language
ILOG Concert Technology (C++, .NET, Java) Java client only
CPLEX Optimization Studio
CPLEX
Enterprise
Server
Connectors
•Microsoft Excel
•MATLAB
•Pytthon
•AMPL
Tools & APIs
•CPLEX Interactive
•C Callable Library
© 2013 IBM Corporation
Decision Optimization
8 INFORMS 2013 Workshop - Innovations
New features
CPLEX 12.5.1 – June 2013– Remote Object, a distributed algorithm toolkit for Java and C++– Additional OPL scripting methods and APIs
CPLEX 12.6 – December 2013– Additional OPL scripting methods– Improved model editor– LP file viewer– New constraints in CP Optimizer– CP Optimizer model dump facility– Global solution to nonconvex QP and MIQP– Distributed MIP algorithm
© 2013 IBM Corporation
Decision Optimization
9 INFORMS 2013 Workshop - Innovations
Additional OPL scripting methods and API
12.5.1 APIs– Relaxation iterator IloOplRelaxationIterator– Conflict iterator IloOplConflictIterator
12.5.1 scripting– Multiple MIP starts
12.6 scripting– Lazy constraints and user cuts– Solution pools– Priorities– Problem queries
© 2013 IBM Corporation
Decision Optimization
10 INFORMS 2013 Workshop - Innovations
Improved model editor
Existing features– Scope selection– Syntax coloring– Search/replace– Syntax errors marked at run time– Etc.
New syntax-aware editing enables– Highlighting occurrences– Renaming per entity type– Refactoring and formatting– Search/find per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing
© 2013 IBM Corporation
Decision Optimization
11 INFORMS 2013 Workshop - Innovations
Improved model editor
New syntax-aware editing enables– Highlighting occurrences– Refactoring and formatting– Search/replace per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing
© 2013 IBM Corporation
Decision Optimization
12 INFORMS 2013 Workshop - Innovations
Improved model editor
New syntax-aware editing enables– Highlighting occurrences– Refactoring and formatting– Search/replace per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing
© 2013 IBM Corporation
Decision Optimization
13 INFORMS 2013 Workshop - Innovations
Improved model editor
New syntax-aware editing enables– Highlighting occurrences– Refactoring and formatting– Search/replace per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing
© 2013 IBM Corporation
Decision Optimization
14 INFORMS 2013 Workshop - Innovations
Improved model editor
New syntax-aware editing enables– Highlighting occurrences– Refactoring and formatting– Search/replace per entity type– Auto-completion– Expand/collapse blocks– Syntax errors marked while editing
© 2013 IBM Corporation
Decision Optimization
15 INFORMS 2013 Workshop - Innovations
LP file viewer
© 2013 IBM Corporation
Decision Optimization
16 INFORMS 2013 Workshop - Innovations
New constraints in CP Optimizer
Strong constraint
SameSequence constraint
© 2013 IBM Corporation
Decision Optimization
17 INFORMS 2013 Workshop - Innovations
New constraints in CP Optimizer
Benefits– Reinforces the model propagation between variables where the user knows
that the domain reduction is weak– Avoids the manual enumeration of tuple sets for creating
allowedAssignments– Creates stronger allowedAssignments than those obtained by hand
Processing– Enumerate partial solutions by performing a search on the variables of each
strong constraint, but using domain reduction of the whole model– Add an allowedAssignments constraint over the variables specified in the
strong constraint, with a tuple set corresponding to the partial solutions found in the previous search
– Remove constraints which are no longer needed given that the strongconstraint is now present in the model
Strong(x1, x2 … xn)
© 2013 IBM Corporation
Decision Optimization
18 INFORMS 2013 Workshop - Innovations
New constraints in CP Optimizer
IloIntTupleSet gha(env, 3);
IloIntArray tuple(env, 3);
for (IloInt i = 0; i < n; i++) {
tuple[0] = i;
for (IloInt j = 0; j < n; j++) {
if (i != j) {
tuple[1] = j;
tuple[2] = Game(i, j, n);
gha.add(tuple);
}
}
}
for (IloInt i = 0; i < nbWeeks; i++) {
for (IloInt j = 0; j < nbGamesPerWeek; j++) {
IloIntVarArray vars(env);
vars.add(home[i][j]);
vars.add(away[i][j]);
vars.add(games[i][j]);
model.add(IloAllowedAssignments(env, vars, gha));
}
}
! Best Branches Non-fixed
* 56 2688 0.53s
* 52 7402 0.99s
* 48 19418 2.38s
* 46 102k 11.87s
* 44 109k 12.81s
* 42 195k 20.22s
* 40 218k 22.43s
* 38 239k 24.47s
for (IloInt i = 0; i < nbWeeks; i++) {
for (IloInt j = 0; j < nbGamesPerWeek; j++) {
model.add(home[i][j] != away[i][j]);
model.add(games[i][j] == Game(home[i][j], away[i][j], n));
IloIntVarArray vars(env);
vars.add(home[i][j]);
vars.add(away[i][j]);
vars.add(games[i][j]);
model.add(IloStrong(env, vars));}
}
! Best Branches Non-fixed W Branch decision
* 50 520 0.44s 6 -
* 46 2468 0.65s 1 -
* 38 4304 0.88s 1 -
* 36 25689 2.98s 1 -
* 34 86973 8.15s 1 -
The original tuple set has 90 tuples, the generated ones have between 5 and 66 tuples.
Using the strong constraint in sports.cpp
© 2013 IBM Corporation
Decision Optimization
19 INFORMS 2013 Workshop - Innovations
New constraints in CP Optimizer
CP Optimizer has the concept of an interval sequence. In OPL:dvar sequence seq1[r in Res] in all(t in tasksOnRes[r]) a[t];
The sameSequence constraint constrains two different sequences to have the same order of corresponding intervalssameSequence(seq1, seq2)
The correspondence can be specified explicitly or be implicit from the order of intervals given in the definition of the sequences
The sameSequence constraint can naturally model constraints like no-bypass whereby products must be processed by different machines in the same order (although that order is free at the outset
SameSequence (seq1, seq2)
M
1
M
2Conveyor belt
seq1 seq2
sameSequence(seq1, seq2)
© 2013 IBM Corporation
Decision Optimization
20 INFORMS 2013 Workshop - Innovations
New constraints in CP Optimizer
dvar interval itvs[j in Jobs][m in Mchs] size
OpDurations[j][m];
dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m]
types all(j in Jobs) j;
minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);
subject to {
forall (m in Mchs)
noOverlap(mchs[m]);
forall (j in Jobs, o in 0..nbMchs-2)
endBeforeStart(itvs[j][o], itvs[j][o+1]);
forall (m in Mchs, j in Jobs: 0<m)
typeOfPrev(mchs[m],itvs[j][m],nbJobs)
== typeOfPrev(mchs[0],itvs[j][0],nbJobs);
}
Using sameSequence in sched_flowshop.mod
dvar interval itvs[j in Jobs][m in Mchs] size OpDurations[j][m];
dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m];
minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);
subject to {
forall (j in Jobs, o in 0..nbMchs-2)
endBeforeStart(itvs[j][o], itvs[j][o+1]);
forall (m in Mchs)
noOverlap(mchs[m]);
forall (m in Mchs: 0<m)
sameSequence(mchs[0], mchs[m]);
}
© 2013 IBM Corporation
Decision Optimization
21 INFORMS 2013 Workshop - Innovations
CP Optimizer model dump facility
Model instance can be written to a file– Constraints– Variables– Search phases– Parameter settings
Easier way to communicate instance to IBM Support
Aids in debugging formulation
Parameter to anonymize names
© 2013 IBM Corporation
Decision Optimization
22 INFORMS 2013 Workshop - Innovations
CPLEX Optimizer parameter hierarchy
Provide consistent naming across APIs
Traditional names will be maintained
Easier to find relevant parameters
Based on hierarchy used in the CPLEX Interactive Optimizer
© 2013 IBM Corporation
Decision Optimization
23 INFORMS 2013 Workshop - Innovations
CPLEX Optimizer parameter hierarchy
API Traditional Hierarchical
Callable Library
CPX_PARAM_FRACCUTS CPXPARAM_MIP_Cuts_Gomory
Java IloCplex.IntParam.FracCuts IloCplex.Param.MIP.Cuts.Gomory
.NET Cplex.IntParam.FracCuts Cplex.Param.MIP.Cuts.Gomory
C++ IloCplex::FracCuts IloCplex::Param::MIP::Cuts::Gomory
Python cplex.parameters.mip.cuts.gomory cplex.parameters.mip.cuts.gomory
MATLAB cplex.Param.mip.cuts.gomory cplex.Param.mip.cuts.gomory
Example
© 2013 IBM Corporation
Decision Optimization
24 INFORMS 2013 Workshop - Innovations
Global solution to nonconvex QP and MIQP
When Q is positive-semidefinite, the objective function is convex
When Q is indefinite, the objective function is nonconvex
min ½ xTQx + cTx
s.t. Ax = b
u ≤ x ≤ l
optionally some x integral
Quadratic program (QP) and
mixed integer quadratic program (MIQP)
© 2013 IBM Corporation
Decision Optimization
25 INFORMS 2013 Workshop - Innovations
Global solution to nonconvex QP and MIQP
Solution to convex QP and MIQP in CPLEX since v. 4.0 and v. 8.0, resp.
Solution to first order optimality for nonconvex QP since v. 12.3– Primal-dual interior point algorithm– Requires linear algebra routines for indefinite factorization – CPLEX wrote own parallel factorization and solution routines
Solution to global optimality for nonconvex QP and MIQP in v. 12.6
© 2013 IBM Corporation
Decision Optimization
26 INFORMS 2013 Workshop - Innovations
Global solution to nonconvex QP and MIQP
Global solution requires branching on continuous variables– QP solved as MIQP, even if no integer variables– NP-complete, can be much more time-consuming than solving to local optimum
Solution Algorithm– Spatial branch-and-cut over convex QP relaxation
• Branch on continuous variables• Tighten linear relaxation in local nodes by locally valid cuts
– Automatic selection of linearization of non-convex part in Q• McCormick relaxation• Eigenvector reformulation based on indefinite factorization
– Solve the indefinite problem when an incumbent is found
© 2013 IBM Corporation
Decision Optimization
27 INFORMS 2013 Workshop - Innovations
Agenda
New features in ILOG CPLEX Optimization Studio
Distributed computing options for CPLEX Optimizer
Performance
© 2013 IBM Corporation
Decision Optimization
28 INFORMS 2013 Workshop - Innovations
Distributed computing*
Core speed vs. more cores
Memory bandwidth on a single system limited by single cache and memory bus
High performance servers
Clusters of commodity hardware
Cloud
*Koch, Shinano and Ralphs. What Could a Million Cores Do To Solve Integer Programs? 2012.
© 2013 IBM Corporation
Decision Optimization
29 INFORMS 2013 Workshop - Innovations
Distributed computing
Client-server– CPLEX Enterprise Server or ILOG ODM Enterprise Optimization Server– CPLEX Remote Object
Distributed algorithms– Build your own with the CPLEX Remote Object– For MIP, use the CPLEX distributed MIP algorithm, new in v. 12.6
© 2013 IBM Corporation
Decision Optimization
30 INFORMS 2013 Workshop - Innovations
Distributed computing: Client-server
Optimization Server in ODM Enterprise or CPLEX Enterprise Server– Built to enterprise-class standards– WebSphere Application Server
• Queuing, failover, etc.• Administration
– Oracle, DB2 or Microsoft SQL database to store results– J2EE compliant– Asynchronous, e.g., client can be disconnected
CPLEX Enterprise Server– From IDE
• IDE panel to submit/monitor/control jobs• Results are returned to IDE
– From Java APIs– OPL .mod and .dat files are sent to server
• Data read on server, not client• Data can be in .dat file or from a database, flat files, etc.
ODM Enterprise– With the Java API for CPLEX, CP Optimizer or OPL via Optimization Server custom task
© 2013 IBM Corporation
Decision Optimization
31 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX Remote Object
Transparent use of remote hardware
Change single line of code to describe connection
All methods are mapped to messages to remote system
Transport abstraction hides the details of using these protocols– ssh– TCP/IP– MPI– Local
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}
© 2013 IBM Corporation
Decision Optimization
32 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX Remote Object
Synchronous or asynchronous
Asynchronous calls support a master/worker distributed parallel programming paradigm for building distributed parallel optimization algorithms
Asynchronous calls return AsyncHandleobjects to join back
Examples provided:– iloparbenders.cpp
– iloparmipopt.cpp
– ParBenders.java
– parbenders.c
– parmipopt.c
optimize(Data data)
{
IloEnv env;
AsyncHandle handle;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
handle = cplex.solve();
doOtherThings();
handle.join();
useSolution(cplex);
}
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}
© 2013 IBM Corporation
Decision Optimization
33 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX distributed MIP
Built on the CPLEX Remote Object technology
A “master” distributes work to multiple “workers”
Two phases– Racing ramp-up*– Each machine uses different settings, and a winner is selected
• Only incumbent objective values and best bounds are communicated• Infinite ramp-up allowed (concurrent MIP)
– Distributed tree• Nodes of the tree created by the winner distributed to workers• Nodes exchanged at synchronization points
– Deterministic
Available in the CPLEX Interactive, and the C, C++, Java, Matlab and Python APIs
*Shinano, Achterberg, Berthold, Heinz, Koch. ParaSCIP – a parallel extension of SCIP. 2010.
© 2013 IBM Corporation
Decision Optimization
34 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX distributed MIP
Does hardware need to be identical?– No, but workers should be similar in performance to get best overall performance– Master does not have to be similar to workers
When should I use infinite ramp-up?– To exploit performance variability– In order to achieve a 1.5x to 2x speed-up for problems which aren’t too trivial
When should I use distributed search?– Large number of nodes to enumerate– Very hard problems
Do I have to have exclusive access to the machine?– No, but for best overall performance, the worker should have little load
Does it work on HPC systems?– Yes. MPI is the de facto standard for HPC and is one of the supported transport
protocols– Experiments have been done on an IBM Blue Gene Q with 256 workers
FAQs
© 2013 IBM Corporation
Decision Optimization
35 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX distributed MIP
Why would I want determinism?– The same solution will be reached each time so the downstream
processing will see consistent results– For tuning and debugging of the application
Which MPI distributions are supported?– OpenMPI and MPICH, depending on the platform
What shared memory parallel MIP features are supported?– MIPstarts – Informational callback on the master– Almost all parameters, e.g.
• Gap• Time limits
– Terminate function
FAQs
© 2013 IBM Corporation
Decision Optimization
36 INFORMS 2013 Workshop - Innovations
Distributed computing: CPLEX distributed MIP
What shared memory parallel MIP features are not supported?– Control callbacks – Restarts– Global solution to nonconvex MIQP and QP
What do I need to do to set up a distributed run?– CPLEX Interactive on each machine with appropriate shared libraries– Describe the machines in a VMC file or with function arguments– Other steps depend on the transport
When does ramp-up stop?– Controlled via parameters– “Automatic” value balances number of open nodes and solution quality
FAQs
© 2013 IBM Corporation
Decision Optimization
37 INFORMS 2013 Workshop - Innovations
Agenda
New features in ILOG CPLEX Optimization Studio
Distributed computing options for CPLEX Optimizer
Performance
© 2013 IBM Corporation
Decision Optimization
38 INFORMS 2013 Workshop - Innovations
Performance – CP Optimizer 12.6
Contains a new scheduling algorithm
Increases search robustness on medium-sized problems
Provides more optimality proofs on smaller instances
The new search is activated by default
Also some performance increase on integer problems
Version-to-version average solution time ratios12.3/12.2 12.4/12.3 12.5/12.4 12.5.1/12.5 12.6/12.5.1
Scheduling1 worker
1.06 1.01 1.29 1.03 1.69
Scheduling4 workers
1.22 0.97 1.35 1.03 1.63
Integer prob.1 worker
1.10 0.99 0.97 1.05 1.10
Integer prob.4 workers
1.06 1.01 1.01 1.03 1.08
© 2013 IBM Corporation
Decision Optimization
39 INFORMS 2013 Workshop - Innovations
Performance – CPLEX Optimizer 12.6
Average speedup of MIP optimizer on MILP v. 12.6 compared to v. 12.5.1– 15% on hard MILP models (more than 100 sec.)– 5% on non-trivial MILP problems (more than 1 sec.)
© 2013 IBM Corporation
Decision Optimization
40 INFORMS 2013 Workshop - Innovations
0
200
400
600
800
1000
1200
1400
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
nu
mb
er o
f ti
meo
uts
0
50
100
150
200
250
tota
l sp
eed
up
10 sec
100 sec
1000 sec
Date: 28 September 2013
Testset: 3147 models (1792 in 10sec, 1554 in 100sec, 1384 in 1000sec)
Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threads (deterministic since CPLEX 11.0)
Timelimit: 10,000 sec
v6.0
v6.5.3
v7.0v8.0
v9.0
v10.0
v11.0 v12.1v12.2
v12.4v12.5
v12.6
CPLEX MILP performance
© 2013 IBM Corporation
Decision Optimization
41 INFORMS 2013 Workshop - Innovations
Technical Sessions
Analyzing 12 Years of Progress in CPLEX Sunday, 16:30 - 18:00 Roland Wunderling
Recent Developments in CPLEX Monday, 08:00 - 09:30 Tobias Achterberg
Non-convex Quadratic Programming in CPLEX Tuesday, 11:00 - 12:30 Christian Bliek and Pierre Bonami
Tutorial: Performance Variability in Mixed-integer Programming Tuesday, 8:00 – 9:30 Andrea Lodi and Andrea Tramontani
Software Demonstration: Expert Tips and Tricks for Using CPLEX Optimization Studio…for Distributed Computing Tuesday Oct 08, 16:30 - 17:15
Lift-and-Project Cuts in CPLEX 12.5.1 Wednesday, 13:30 - 15:00 Andrea Tramontani
Concurrent root cut loops to exploit random performance variability Wednesday, 15:30 - 17:00 A. Tramontani, M. Fischetti, A. Lodi, D. Salvignin, M. Monaci
Interesting Use Cases for the CPLEX Remote Object Wednesday, 15:30 - 17:00 Laszlo Ladanyi and Daniel Junglas