Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math...
Transcript of Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math...
![Page 1: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/1.jpg)
Math Models of OR: The Klee-Minty Cube
John E. Mitchell
Department of Mathematical SciencesRPI, Troy, NY 12180 USA
September 2018
Mitchell The Klee-Minty Cube 1 / 23
![Page 2: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/2.jpg)
Visiting all extreme points
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 2 / 23
![Page 3: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/3.jpg)
Visiting all extreme points
Worst-case performance of simplex
The simplex algorithm proceeds from one extreme point to aneighboring extreme point that is at least as good.
How many extreme points can it visit?
The Klee-Minty cube is an example where it visits every extremepoint. It’s a problem that is expressed in terms of n inequalityconstraints on n variables, together with nonnegativity constraints. Thenumber of extreme points is exponential in the size of the problem, 2n.
Thus, simplex has what is called exponential runtime in the worst case.Polynomial runtime (for example, 4n3 iterations) is far preferable: itgrows far more slowly than exponential runtime.
For large scale problems, linear run time (for example, 3n iterations) isdesirable.
Mitchell The Klee-Minty Cube 3 / 23
![Page 4: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/4.jpg)
The Klee-Minty cube in IR3
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 4 / 23
![Page 5: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/5.jpg)
The Klee-Minty cube in IR3
The Klee-Minty cube in 3 dimensions
Consider the linear program
minx∈IR3 −100x1 − 10x2 − x3subject to x1 ≤ 1
20x1 + x2 ≤ 100200x1 + 20x2 + x3 ≤ 10000
x1, x2, x3 ≥ 0
Mitchell The Klee-Minty Cube 5 / 23
![Page 6: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/6.jpg)
The Klee-Minty cube in IR3
The Klee-Minty cube
x1
x2
x3
0 64
(0,0,0)(0,100,0)
(0,0,10000)
(1,0,0)
(1,80,0)
(0,100,8000)
(1,80,8200)
(1,0,9800)
Mitchell The Klee-Minty Cube 6 / 23
![Page 7: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/7.jpg)
The iterations
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 7 / 23
![Page 8: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/8.jpg)
The iterations
Pivot rule
The variable that enters the basis is the nonbasic variable with themost negative reduced cost.
There are no ties in the minimum ratio for this example, so it is notnecessary to specify the tie-breaking mechanism.
Mitchell The Klee-Minty Cube 8 / 23
![Page 9: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/9.jpg)
The iterations
Get initial canonical form
After introducing slack variables, we have a problem in canonical formso we can proceed directly with simplex. The initial basis consists ofthe slack variables, denoted x4, x5, x6.
↓ratio x1 x2 x3 x4 x5 x6
0 −100 −10 −1 0 0 01 1 1© 0 0 1 0 05 100 20 1 0 0 1 0
50 10000 200 20 1 0 0 1
Mitchell The Klee-Minty Cube 9 / 23
![Page 10: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/10.jpg)
The iterations
After one iteration
R0+100R1,R2−20R1,R3−200R1−→
↓ratio x1 x2 x3 x4 x5 x6
100 0 −10 −1 100 0 0− 1 1 0 0 1 0 080 80 0 1© 0 −20 1 0
490 9800 0 20 1 −200 0 1
Mitchell The Klee-Minty Cube 10 / 23
![Page 11: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/11.jpg)
The iterations
After two iterations
R0+10R2,R3−20R2−→
↓ratio x1 x2 x3 x4 x5 x6
900 0 0 −1 −100 10 01 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 041 8200 0 0 1 200 −20 1
Mitchell The Klee-Minty Cube 11 / 23
![Page 12: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/12.jpg)
The iterations
After three iterations
R0+100R1,R2+20R1,R3−200R1−→
↓ratio x1 x2 x3 x4 x5 x6
1000 100 0 −1 0 10 0− 1 1 0 0 1 0 0− 100 20 1 0 0 1 0
8000 8000 −200 0 1© 0 −20 1
Mitchell The Klee-Minty Cube 12 / 23
![Page 13: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/13.jpg)
The iterations
After four iterations
R0+R3−→
↓ratio x1 x2 x3 x4 x5 x6
9000 −100 0 0 0 −10 11 1 1© 0 0 1 0 05 100 20 1 0 0 1 0− 8000 −200 0 1 0 −20 1
Mitchell The Klee-Minty Cube 13 / 23
![Page 14: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/14.jpg)
The iterations
After five iterations
R0+100R1,R2−20R1,R3+200R1−→
↓ratio x1 x2 x3 x4 x5 x6
9100 0 0 0 100 −10 1− 1 1© 0 0 1 0 080 80 0 1 0 −20 1 0− 8200 0 0 1 200 −20 1
Mitchell The Klee-Minty Cube 14 / 23
![Page 15: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/15.jpg)
The iterations
After six iterations
R0+10R2,R3+20R2−→
↓ratio x1 x2 x3 x4 x5 x6
9900 0 10 0 −100 0 11 1 1 0 0 1© 0 0− 80 0 1 0 −20 1 0− 9800 0 20 1 −200 0 1
Mitchell The Klee-Minty Cube 15 / 23
![Page 16: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/16.jpg)
The iterations
After seven iterations
R0+100R1,R2+20R1,R3+200R2−→
x1 x2 x3 x4 x5 x610000 100 10 0 0 0 1
1 1 0 0 1 0 0100 20 1 0 0 1 0
10000 200 20 1 0 0 1
This is optimal, with value −10000 and x1 = 0, x2 = 0, x3 = 10000.
The number of pivots required was 7 = 23 − 1.
Every extreme point was visited.
Mitchell The Klee-Minty Cube 16 / 23
![Page 17: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/17.jpg)
The iterations
All the iterations
x1
x2
x3
(0,0,0)(0,100,0)
(0,0,10000)
(1,0,0)
(1,80,0)
(0,100,8000)
(1,80,8200)
(1,0,9800)
Mitchell The Klee-Minty Cube 17 / 23
![Page 18: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/18.jpg)
Alternative pivot rules
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 18 / 23
![Page 19: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/19.jpg)
Alternative pivot rules
Alternative pivot rules
If the best improvement rule had been used to choose the incomingvariable, x3 would have entered the basis on the first iteration and theproblem would have been solved in one step.
A variant of the Klee-Minty cube has been designed that also requiresexponentially many iterations in the worst-case, even with the bestimprovement rule.
Other variants with exponential worst-case performance have beendesigned for all known pivot rules.
Mitchell The Klee-Minty Cube 19 / 23
![Page 20: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/20.jpg)
Extending to IRn
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 20 / 23
![Page 21: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/21.jpg)
Extending to IRn
Extending to n variables
minx∈IRn −∑n
j=1 10n−jxj
subject to 2∑i−1
j=1 10i−jxj + xi ≤ 100i−1 i = 1, . . . ,nxj ≥ 0 j = 1, . . . ,n
The simplex method choosing the most negative entry to enter thebasis requires 2n − 1 iterations to solve this problem.
Mitchell The Klee-Minty Cube 21 / 23
![Page 22: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/22.jpg)
Average performance
Outline
1 Visiting all extreme points
2 The Klee-Minty cube in IR3
3 The iterations
4 Alternative pivot rules
5 Extending to IRn
6 Average performance
Mitchell The Klee-Minty Cube 22 / 23
![Page 23: Math Models of OR: The Klee-Minty Cubeeaton.math.rpi.edu/faculty/Mitchell/courses/matp4700/...Math Models of OR: The Klee-Minty Cube John E. Mitchell Department of Mathematical Sciences](https://reader034.fdocuments.net/reader034/viewer/2022051810/6017aad2e89a6d2ded082254/html5/thumbnails/23.jpg)
Average performance
Average performance of simplex
There is theoretical analysis showing that the average number ofiterations is linear in the size of the linear optimization problem.
(Need to make assumptions about the distribution of instances.)
This is also the typical performance in practice.
Mitchell The Klee-Minty Cube 23 / 23