Copyright © ILOG 2008 IBM ILOG CPLEX Advanced Performance Tuning Optimization Technologies Workshop...

Copyright © ILOG 2008

IBM ILOG CPLEX Advanced Performance Tuning

Optimization Technologies Workshop

John GregoryCPLEX Product Manager IBM [email protected]

2


CPLEX Performance Tuning Overview

Speed Continuous Models (LP, QP, QCP/SOCP)

MIP

Memory Operating System

Deployment Application

CPLEX

Numerical Ill conditioning

Numerical Instability

Summary

3


Tuning for Speed: Continuous Models

Try each algorithm Primal Simplex

Dual Simplex

Network Simplex

Barrier (only option for QCP/SOCP)

Sifting (LP only)

Concurrentopt (LP only, parallel processors)

4



Other tactics Presolve dual on (apply presolve, then solve the dual model)

LP only

Constraints : 1972170 [Greater: 1972170]

Variables : 275036 [Free: 275036]

Constraint nonzeros: 9234686

Objective nonzeros: 156817

RHS nonzeros: 1487481

Parallel barrier method

Check for numerical difficulties

5



Iteration logs Degeneracy (simplex methods)

Excessive time per iteration

6


Tuning for Speed : Continuous Models

Degeneracy, sample log Iter: 4751 Infeasibility = 8.000000

Iter: 4870 Infeasibility = 8.000000




...



Perturbation started.

7



Excessive time per iteration, sample log

Elapsed time = 138.23 sec. (37000 iterations)





...





12 sec.

827 sec.

8



Degeneracy Try each algorithm

Instruct CPLEX to perturb the model

Excessive time per iteration Try each algorithm

Check for virtual memory usage Run with more memory available Turn on memory emphasis parameter

Consider alternate pricing (pgradient/dgradient) Time per iteration depends on pricing scheme

9


Tuning for Speed: MIP

Try the Tuning Tool (CPLEX 11 or later) Primarily for MIPs, but works for all problem types

Works with all CPLEX interfaces, including interactive

optimizer

Can make use of internal algorithm information

Let CPLEX and your computer do the work

10



Input: Model(s) to tune, tuning instructions Problem in memory or set of problems in files

Fixed parameter set, parameters which should not be

tuned, e.g., tolerances, threads

Parameters, metrics to control tuning process (e.g.

total tuning time, average or worst case run time)

Output: Tuned and fixed parameter settings

11



Examine node log (if Tuning Tool doesn’t help) Excessive iteration counts for node relaxations.

Use different start/sub algorithm. Barrier method doesn’t support restarts. Set dual gradient to exact steepest edge norms, file

parameter to store node file on disk and compress it.

More progress needed in best integer or best node

value?

12


Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt

0 0 -89.0000 6 -89.0000 5278 0 0 -89.0000 6 Fract: 4 12799 0 2 -89.0000 6 -89.0000 12799 1 1 infeasible -89.0000 20767 2 2 -89.0000 5 -89.0000 27275 3 1 infeasible -89.0000 32502 … 8 2 -89.0000 8 -89.0000 65717 9 1 infeasible -89.0000 73714 … Solution time = 177.33 sec. Iterations = 73714 Nodes = 10 (1)


Excessive iteration counts for node LPs, sample

node log #1 (dual simplex to solve node LPs)

13



Excessive iteration counts for node LPs, sample

node log #2 (primal simplex to solve node LPs) Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt 0 0 -89.0000 5 -89.0000 6603 0 0 -89.0000 5 Fract: 5 7120 0 2 -89.0000 5 -89.0000 7120 1 1 infeasible -89.0000 9621 2 2 -89.0000 5 -89.0000 10616 3 1 infeasible -89.0000 12963 … 8 2 -89.0000 8 -89.0000 21522 9 1 infeasible -89.0000 23891 … Solution time = 54.37 sec. Iterations = 23891 Nodes = 10 (1)

14



Progress in best node but not best integer:

Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt Gap... 300 229 22.6667 40 31.0000 22.0000 4433 29.03% 400 309 cutoff 31.0000 22.3333 5196 27.96% 500 387 26.5000 31 31.0000 22.6667 6164 26.88%...

7800 5260 28.5000 23 31.0000 25.6667 55739 17.20% 7900 5324 28.2500 26 31.0000 25.6667 56424 17.20% 8000 5385 27.3750 30 31.0000 25.7778 57267 16.85%

15



Progress in best integer but not best node:

Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt Gap… 300 296 2018.0000 27 3780.0000 560.0000 3703 85.19%* 300+ 296 0 2626.0000 560.0000 3703 78.67%* 393 368 0 2590.0000 560.0000 4405 78.38% 400 372 560.0000 291 2590.0000 560.0000 4553 78.38% 500 472 810.0000 175 2590.0000 560.0000 5747 78.38%...

* 7740+ 5183 0 1710.0000 560.0000 66026 67.25% 7800 5240 1544.0000 110 1710.0000 560.0000 66279 67.25% 7900 5325 944.0000 176 1710.0000 560.0000 66801 67.25% 8000 5424 1468.0000 93 1710.0000 560.0000 67732 67.25%

16



Tactics for lack of progress in best integer Use especially if customer accepts a good, perhaps

sub optimal solution.

MIP emphasis Set to 1 (emphasize integer feasibility) Set to 4 (emphasize finding hidden integer solutions)

Solution Polishing Requires an integer feasible solution Use to improve sub optimal solutions at end of MIP search Or use as replacement for majority of MIP search when

- local search works well- node LPs are time consuming- improving best node appears hopeless

17



Lack of progress in best integer (continued) RINS Heuristic

Local search heuristic- Needs an integer feasible solution

MIP Starts and Solution Repair CPLEX 10.0 and later accepts partial MIP starts

- Solution repair for (partial) infeasible MIP starts

May make RINS, Solution Polishing more effective

Feasibility Pump Heuristic (CPLEX 11 and later) Applied first at root node

18



Lack of progress in best integer (continued) Probing

Set to 3 (maximum probing)

Cuts Set to 2 (aggressive cut generation)

Cuts and Probing don’t explicitly help find feasible

solutions, but the resulting tighter formulation may

help. Can be useful when combined with setting MIP emphasis to

1 (emphasize integer feasibility)

19



Tactics for lack of progress in best node MIP emphasis

Set to 2 or 3 to emphasize proving optimality and moving best node respectively.

- May need Partial/Infeasible MIP start, Feasibility Pump or RINS Heuristic to compensate for fewer integer feasible solutions

Probing Set to 3 (maximum probing)

Cuts Set to 2 (aggressive cut generation)

Cuts and Probing often tighten the model formulation

20



Tactics for lack of progress in best node and

best integer Set MIP emphasis to 3

May cause difficulties finding integer feasible solutions Compensate by using (partial or infeasible) MIP starts,

Feasibility Pump, RINS Heuristic, Solution Polishing to help find more integer feasible solutions

21



Tactics for lack of progress in best node and

best integer (continued) Provide a starting solution, regardless of objective

value Can help CPLEX’s local search heuristics Can yield additional tightening of the model Consider external methods (e.g. Constraint Programming,

Metaheuristics like Simulated Annealing, model specific heuristics) to find a starting solution

Try CPLEX’s feasopt feature

22



Other tactics Specific model knowledge determines parameter

settings

Try branching up first

1

1

i

n

i

x binary ix

23



Other tactics (continued) Specific model knowledge determines priority order.

Time period models- Higher priority to earlier time periods.

Dependencies among integer variables

0 - ii Uzxinteger ix binary iz

24



Other tactics (continued) Specific model knowledge determines changes to

model Adding model specific cuts

- Tighten formulation- Remove symmetry

Sherali, Smith “Improving Zero-One Model Representations Via Symmetry Considerations”

Model reformulation http://www.ilog.com/products/cplex/news/newsletters/

cplex_newsletter_9410.cfm

25


Memory Tuning: Operating System

Virtual memory slows performance Task Manager (Windows)

Top, ps, vmstat under Unix

Make sure you can allocate all available memory Java VM limitations

Use –Xmx option to increase heap size

On a 32-bit machine, the real limit is 2GB

Limits on individual accounts

Compiler/Linker options 32 bit AIX has default of 256 MB for memory allocations.

Write a simple program to test available memory (or ask us for one).

26


Memory Tuning: Deployment Considerations

Compare memory usage Solve the Deployment application model(s) with:

Interactive CPLEX Simple example programs

Purify, Insure, Bounds Checker, Valgrind (Linux only) can help with memory leaks

Make sure you solve the correct model Display problem statistics

Data check parameter

Diagnostic routines in check.c

27


Memory Tuning: CPLEX Considerations

Sometimes CPLEX really uses all available

memory Default settings emphasize speed over memory

conservation.

Is CPLEX memory usage reasonable? 1 GB per million constraints of an LP

- Exceptions: large aspect ratios, very dense constraints

1 GB per million (constraints + quadratic variables) of a QP- Exception: dense Q matrix lacking sparse factorization

At least that much for MIP, MIQP

28



Memory conservation techniques General

Turn memory emphasis on (slight extra run time) Use different algorithm (LP and QP) As a last resort, turn presolve off (potential big increase in

run time)

LP Set presolve dual indicator on

29



Memory conservation techniques (continued) MIP

MIP search tree often dominates memory usage- Performance tuning to reduce node count.- Setting node file parameter to 3 efficiently writes nodes to

disk.- Setting variable select parameter to 3 (strong branching).

Fewer nodes required to make the same amount of progress.

30


Memory Tuning: Other Tactics

Buy more memory (but only after accurately

estimating requirements)

Specialized algorithms Example: Column generation for models with large

aspect ratios.

31


Memory Tuning: Other Tactics

Review of model development process

Model reformulation Reduce problem dimensions

Reduce “difficulty” while increasing problem size. MIP reformulations for column generation. Increase in problem size in exchange for sparser problem

data

st min QxxT

bAx st min yyT

0 LxybAx

LLQ T

32


Numerical Tuning: Ill conditioning

Sensitivity of computed solutions to perturbations in problem data

Condition number of a matrix CPLEX solves systems of equations of the form

Exact solution is

bBx

bBx 1

33



Condition number of a matrix (continued) To measure effects of perturbations, consider

Exact solution is

Combine with exact solution of x:

bbxxB )(

)(1 bbBxx

bBx 1

34



Condition number of a matrix (ctd). Using Cauchy - Schwarz inequality

This gives max. absolute change in x relative to

change in b. Now consider relative change by using

Cauchy - Schwarz inequality in original system:

|| || |||| |||| xBb

|| || |||| |||| 1 bBx

35



Condition number of a matrix (continued) Combine and rearrange to get

Similar procedure for perturbing B yields

||)|| / || (|| |||| |||| || || / |||| 1 BBBBxxx

|| || || || )cond( 1 BBB

||)|| / || (|| |||| |||| || || / |||| 1 bbBBxx

36



Ill conditioning can occur under perfect

arithmetic

But, finite precision can perturb the exact

system of equations we wish to solve, resulting

in significant changes to the computed solution

37



Sources of perturbations due to finite precision Rounding of data

Near singular matrices Avoid computing data in single precision

Round off error in the simplex method computations,

particularly the basis factorization.

Round off error due to scaling of large data values

Floating point calculations involving values of greatly

different orders of magnitude Avoid mixtures of very large and small coefficients

38



Rounding of data

Maximize x1 + x2 c1: 1/3 x1 + 2/3 x2= 1 c2: x1 + 2 x2 = 3 Solution: x1 = 3.0

Maximize x1 + x2 c1: .333333 x1 + .666667 x2 = 1 c2: x1 + 2 x2 = 3 Solution: x1 = x2 = 1.0, cond(B) = 6.66667e+06

Maximize x1 + x2 c1: .333333333 x1 + .666666667 x2 = 1 c2: x1 + 2 x2 = 3 Solution: x1 = 3.0, cond(B) = 6.333333

39



Avoid rounding whenever possible

Exact formulation:Maximize x1 + x2 c1: 1/3 x1 + 2/3 x2= 1 c2: x1 + 2 x2 = 3

Imprecisely rounded:Maximize x1 + x2 c1: .333333 x1 + .666667 x2 = 1 c2: x1 + 2 x2 = 3

Scale to integral value whenever possible:Maximize x1 + x2 c1: x1 + 2 x2 = 3 c2: x1 + 2 x2 = 3

40



Rounding of data (continued) Changes in right hand side can have similar effect

Maximize x1 + x2 c1: .333333 x1 + .666667 x2 = 1 c2: x1 + 2 x2 = 3 Solution: x1 = x2 = 1.0

Maximize x1 + x2 c1: .333333 x1 + .666667 x2 = .999999 c2: x1 + 2 x2 = 3 Solution: x1 = 3.0

41


Numerical Tuning: Numerical Instability

Large values in problem data Bounds can magnify round off error:

Large coefficient ratios in matrix rows or columns:

0 z 01000000000 xbinary 5000, zx

NNNB xAAbBx ) ( -

42


Symptoms of Numerical Instability

Sample iteration log

Iter: 6389 Objective = 13137.039899Iter: 6390 Objective = 13137.039899Iter: 6391 Objective = 13726.011591Markowitz threshold set to 0.3.Iter: 6456 Scaled infeas = 300615.030682...Iter: 6752 Scaled infeas = 0.000002Iter: 6754 Objective = -23870.812630

43


Numerical Tuning Tactics

General Turn numerical emphasis on

Rounding of data Compute data as accurately as possible

Round off error in simplex method Set Markowitz tolerance to .90 (or higher if

necessary)

Large values in problem data Reduce large coefficients if possible

Set scaling parameter to 1 (aggressive scaling)

44


CPLEX Performance Tuning Summary

CPLEX solves most problems with no trouble Organized tactics for the outliers can yield dramatic savings in

time.

Different categories of performance problems.

Use all available information Iteration and node logs for LP and MIP

Interactive CPLEX

Simple test programs (CPLEX and your own)

OS memory management tools

Purify/Insure/Bounds Checker/Valgrind

ILOG Technical Support and Consulting

Copyright © ILOG 2008 IBM ILOG CPLEX Advanced Performance Tuning Optimization Technologies Workshop...

Documents

Transcript of Copyright © ILOG 2008 IBM ILOG CPLEX Advanced Performance Tuning Optimization Technologies Workshop...