Post on 10-Oct-2018
Introduction to Operations Research(Week 6: Linear Programming:
Duality and More on Post-Optimality)
Jose Rui Figueira
Instituto Superior TecnicoUniversidade de Lisboa
(figueira@tecnico.ulisboa.pt)
March 28-29, 2016
This slides are currently available for personal use of IST LEGI FIOstudents in an unpublished draft form only. The slides cannot becopied, reproduced, or distributed in any form.
Part
Duality
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
Contents
1. Introduction
2. Building the dual from an example
3. Solving the primal solves the dual and vice-versa
4. Transformation rules
5. The Dual (generalization)
6. Main results
7. More relations between primal and dual
8. An exercise
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
1. Introduction
Brief Introduction
I How to build the Dual of a Primal LP?
I Why is a Dual LP so important?
I It allows to have a different view of the problem.
I It allows to help in the economic interpretation of the problem.
I It allow to develop new mathematical concepts for LP.
I It allows the design of very efficient algorithms.
I References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989,Hillier and Lieberman, 2005, Taha, 2010, Williams, 1993, Williams, 1999]
J.R. Figueira (IST) FIO March 28-29, 2016 5 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
The Primal LP Problem
max z(x1, x2) = 3x1 + 4x2 Profit
subject to: 2x1 + x2 6 18 (y1) 1 Silicone
x1 + 2x2 6 15 (y2) 2 High-tech machine
x1 6 8 (y3) 3 Robot 1
x2 6 6 (y4) 4 Robot 2
x1 > 0 Nonnegativity of x1
x2 > 0 Nonnegativity of x2
There are always three components to take into account for building the dual:
I The objective function, which is a minimization (in the dual) in this case.
I Associate with each constraint of the Primal there is a Dual variable (4 dualvariables).
I Associate with each Primal variable there is a Dual constraint (2 dualconstraints).
J.R. Figueira (IST) FIO March 28-29, 2016 6 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3
y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4
y1 > 0y2 > 0
y3 > 0y4 > 0
I Objective function of the Dual: The coefficients of the objective function of theDual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0
y3 > 0y4 > 0
I Objective function of the Dual: The coefficients of the objective function of theDual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0
I Objective function of the Dual: The coefficients of the objective function of theDual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0
I Objective function of the Dual: The coefficients of the objective function of theDual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
Building the Dual
min w(y1, y2, y3, y4) = 18y1 + 15y2 + 8y3 + 6y4
subject to: 2y1 + y2 + y3 > 3y1 + 2y2 + y4 > 4y1 > 0
y2 > 0y3 > 0
y4 > 0I Objective function of the Dual: The coefficients of the objective function of the
Dual are the RHS (bi) of the Primal constraints.
I Constraints of the Dual: The RHS of the Dual constraints are the coefficients ofthe decision variables (cj) of the Primal objective function.
I Each Dual constraint is built from each Primal activity: The LHS coefficients of theDual constraint (j) are the technological coefficients (aij) for the Primal activity (j).
I Analyze, for example, the relation between c1 = z1 − c1 and the first constraint ofthe dual LP? 2y1 + y2 + y3 > 3 ⇔ (2y1 + y2 + y3)− 3 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 7 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3
y1 + 2y2 + y4 − y6 = 4y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
2. Building the dual from an example
The Dual LP in an equality form
min w(y) = 18y1 + 15y2 + 8y3 + 6y4 − 0y5 − 0y6
subject to: 2y1 + y2 + y3 − y5 = 3y1 + 2y2 + y4 − y6 = 4
y1, y2, y3, y4, y5, y6 > 0
I Decision variables of the Dual: y1, y2, y3, and y4.
I Surplus variables of the Dual: y5 and y6.
I Since solving the Primal also solves the Dual, where are the values of theses Dualvariables?
I Let us have a look at the optimal primal solution (in the last Tableau).
J.R. Figueira (IST) FIO March 28-29, 2016 8 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
3. Solving the primal solves the dual and vice-versa
cj 3 4 0 0 0 0Basis xj x1 x2 x3 x4 x5 x6 bi Ratio
0 x6 0 0 1/3 −2/3 0 1 23 x1 1 0 2/3 −1/3 0 0 70 x5 0 0 −2/3 1/3 1 0 14 x2 0 1 −1/3 2/3 0 0 4
t = 3 zj 3 4 2/3 5/3 0 0D zj − cj 0 0 2/3 5/3 0 0 37
I Consider the optimal primal simplex Tableau.
I The Dual solution is in lines zj and cj = zj − cj.
I The Dual decision variables are related with the primal constraints and so with theprimal slack variables. Why?
I Their values are in the line zj under the slack variables of the primal problem. Why?
I Thus, y1 = 2/3, y2 = 5/3, y3 = 0, and y4 = 0. Why?
I What about the values of y5 and y6?
J.R. Figueira (IST) FIO March 28-29, 2016 9 / 45
4. Transformation rules
Objective, Constraints, and Variables Types
Primal Dual
Objective Function � Objective Functionmax min
Variables � Constraints> 0 >6 0 6free =
Constraints � Variables6 > 0> 6 0= free
J.R. Figueira (IST) FIO March 28-29, 2016 10 / 45
5. The Dual (generalization)
The LP Dual Problem
minm
∑i=1
biyi
subject to:m
∑i=1
aijyi > cj, j = 1, . . . , n,
yi > 0, i = 1, . . . , m.
Or, by adding the surplus (“slack”) variables, ym+1, . . . , ym+j, . . . , ym+n:
minm
∑i=1
biyi
subject to:m
∑i=1
aijyi − ym+j = cj, j = 1, . . . , n,
yi > 0, i = 1, . . . , m + n.
J.R. Figueira (IST) FIO March 28-29, 2016 11 / 45
5. The Dual (generalization)
The dual as the primal in the “mirror”
c1...
cj...
cnm
inw
>...
>...
>=
a11...
a1j...
a1nb1
(y1 )
......
......
......
ai1...
aij...
ainbi
(yi )
......
......
......
am1
...am
j...
amn
bm(y
m)
(x1) (xj) (xn)c1 . . . cj . . . cn = max za11 . . . a1j . . . a1n 6 b1
.... . .
.... . .
......
...ai1 . . . aij . . . ain 6 bi...
. . ....
. . ....
......
am1 . . . amj . . . amn 6 bm
DUAL
(yi > 0, i = 1, . . . , m)PRIMAL
(xj > 0, j = 1, . . . , n)
J.R. Figueira (IST) FIO March 28-29, 2016 12 / 45
6. Main results
Symmetry Between Primal and Dual.
maxx
{c>x : Ax ≤ b, x ≥ 0
}(P)
miny
{b>y : A>y ≥ c, y ≥ 0
}(D)
and
max(x,s)
{c>x : Ax + s = b, (x, s) ≥ 0
}(P′)
min(y,r)
{b>y : A>y− r = c, (y, r) ≥ 0
}(D′)
where s = (xn+1, . . . , xn+m)> and r = (ym+1, . . . , ym+n)> are the vectors ofslack variables of the primal and the dual problems, respectively.
J.R. Figueira (IST) FIO March 28-29, 2016 13 / 45
6. Main results
Two Lemmas on Duality
If (D) is the dual of a problem (P), then the dual of (D) is (P).
Lemma 1 (The Dual of the Dual).
Any feasible solution of (D) yields an upper bound for (P).
Lemma 2 (Upper Bound).
J.R. Figueira (IST) FIO March 28-29, 2016 14 / 45
6. Main results
Important Theorems on Duality
Let x ∈ Rn denote a feasible solution for (P) and y ∈ Rm denote a feasiblesolution for (D). Then, the following relation holds: c>x 6 b>y.
Theorem 1 (Weak Duality).
Let x∗ ∈ Rn denote a feasible solution for (P) and y∗ ∈ Rm denote afeasible solution for (D). If c>x∗ = b>y∗, then x∗ and y∗ are the optimalsolutions of (P) and (D), respectively.
Theorem 2 (Strong Duality).
J.R. Figueira (IST) FIO March 28-29, 2016 15 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.
J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.
J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.
J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.
J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.
J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Proving Strong Duality: Farka’s Lemma (A variant)
Let A ∈ Rm×n and let b ∈ Rm. Then exactly one of the two alternativesholds:
1. There exists x ≥ 0, such that Ax = b.
2. There exists y, such that y>A ≥ 0 and y>b < 0.
Lemma 3 (Farka’s Lemma).
This result simply states
1. that a vector is either in a given convex cone, or [it is an exclusive or]
2. that there exists a hyperplane separating the vector from the cone.
(there are no other possibilities.)
Let Ai denote the i− th column of matrix A. Next slide shows a geometric view of this
lemma.J.R. Figueira (IST) FIO March 28-29, 2016 16 / 45
6. Main results
Geometric Illustration of Farka’s Lemma
A1 A2 A3
b
b
y
Hy,0
J.R. Figueira (IST) FIO March 28-29, 2016 17 / 45
6. Main results
Geometric Illustration of Farka’s Lemma
A1 A2 A3
b
b
y
Hy,0
J.R. Figueira (IST) FIO March 28-29, 2016 17 / 45
7. More relations between primal and dual
A Different Optimality Condition
Let (x∗, s∗) ∈ Rn ×Rm denote a feasible solution for (P′) and (y∗, r∗) ∈Rm ×Rn denote a feasible solution for (D′). Then x∗ and y∗ are optimalfor (P) and (D), respectively iff x∗>r∗ = 0 and y∗>s∗ = 0.
Theorem 3 (Complementary Slackness).
J.R. Figueira (IST) FIO March 28-29, 2016 18 / 45
7. More relations between primal and dual
Possible results for (P) and (D)
Exactly one of the following statements is true:
1. Both Problem (P) and Problem (D) possess optimal solutions x∗ andy∗, respectively, and c>x∗ = b>y∗.
2. Problem (P) is unbounded and Problem (D) is infeasible.
3. Problem (D) is unbounded and Problem (P) is infeasible.
4. Both problems are infeasible.
Theorem 4 (Fundamental Theorem of Primal-Dual Relationships).
J.R. Figueira (IST) FIO March 28-29, 2016 19 / 45
8. An exercise
Linear Programming (LP)
What is the dual of the following primal LP problem?
maxn
∑j=1
cjxj
subject to:n
∑j=1
aijxj 6 bi, i = 1, . . . , m′ ,
n
∑j=1
aijxj > bi, i = m′ + 1, . . . , m′′ ,
n
∑j=1
aijxj = bi, i = m′′ + 1, . . . , m′′′ ,
with xj > 0, for all j = 1, . . . , n.
Exercise 1 (Dual problem).
J.R. Figueira (IST) FIO March 28-29, 2016 20 / 45
Part
Economic Interpretation
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
Contents
1. Introduction
2. Product-mix resource allocation problem
3. The optimal primal simplex Tableau
4. What are the shadow prices?
5. Another view of the problem
6. More on the economic interpretation
7. Lagrangian Duality
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
1. Introduction
Brief Introduction
I We start again with the resource allocation problem.
I We will see a new view of the problem.
I The relations between dual variables, internal value of resources, shadowprices, and ...
I A different view of duality.
J.R. Figueira (IST) FIO March 28-29, 2016 23 / 45
2. Product-mix resource allocation problem
A Classical Maximization Problem
The CorkMaxPro Company is the largest Portuguese producer andexporter of cork made products. Over the last five years, the com-pany has gradually expanded the geography and the volume of itsproduction activities, which consist of producing a product mix (i.e., de-signing and making the product lines or assortments that the companyoffers to its clients), here generically denoted by P1, . . ., Pj, . . ., and Pn.
The company’s plants possess several resources, R1, . . ., Ri, . . ., andRm, to make the different assortments. The quantity available of eachresource is, b1, . . ., bi, . . ., and bm, respectively. The allocation of resourcesto the production is as follows: allocate aij units of the resource Ri to makeone unit of the assortment Pj.
The market absorbs all the production and the revenue per unit ofproduct sold is cj Euros. The Company’s CEO needs to know the next yearproduction plan (activity levels) in such a way that the resource constraintsshould be taken into account and the overall revenue to be earned from theselling of the assortments in the market should be maximized.
Example 1 (A Classical Product Mix Resource Allocation Problem).
J.R. Figueira (IST) FIO March 28-29, 2016 24 / 45
2. Product-mix resource allocation problem
A Classical Maximization Problem
The CorkMaxPro Company is the largest Portuguese producer andexporter of cork made products. Over the last five years, the com-pany has gradually expanded the geography and the volume of itsproduction activities, which consist of producing a product mix (i.e., de-signing and making the product lines or assortments that the companyoffers to its clients), here generically denoted by P1, . . ., Pj, . . ., and Pn.
The company’s plants possess several resources, R1, . . ., Ri, . . ., andRm, to make the different assortments. The quantity available of eachresource is, b1, . . ., bi, . . ., and bm, respectively. The allocation of resourcesto the production is as follows: allocate aij units of the resource Ri to makeone unit of the assortment Pj.
The market absorbs all the production and the revenue per unit ofproduct sold is cj Euros. The Company’s CEO needs to know the next yearproduction plan (activity levels) in such a way that the resource constraintsshould be taken into account and the overall revenue to be earned from theselling of the assortments in the market should be maximized.
Example 1 (A Classical Product Mix Resource Allocation Problem).
J.R. Figueira (IST) FIO March 28-29, 2016 24 / 45
2. Product-mix resource allocation problem
A Classical Maximization Problem
The CorkMaxPro Company is the largest Portuguese producer andexporter of cork made products. Over the last five years, the com-pany has gradually expanded the geography and the volume of itsproduction activities, which consist of producing a product mix (i.e., de-signing and making the product lines or assortments that the companyoffers to its clients), here generically denoted by P1, . . ., Pj, . . ., and Pn.
The company’s plants possess several resources, R1, . . ., Ri, . . ., andRm, to make the different assortments. The quantity available of eachresource is, b1, . . ., bi, . . ., and bm, respectively. The allocation of resourcesto the production is as follows: allocate aij units of the resource Ri to makeone unit of the assortment Pj.
The market absorbs all the production and the revenue per unit ofproduct sold is cj Euros. The Company’s CEO needs to know the next yearproduction plan (activity levels) in such a way that the resource constraintsshould be taken into account and the overall revenue to be earned from theselling of the assortments in the market should be maximized.
Example 1 (A Classical Product Mix Resource Allocation Problem).
J.R. Figueira (IST) FIO March 28-29, 2016 24 / 45
2. Product-mix resource allocation problem
A Classical Maximization Problem
The CorkMaxPro Company is the largest Portuguese producer andexporter of cork made products. Over the last five years, the com-pany has gradually expanded the geography and the volume of itsproduction activities, which consist of producing a product mix (i.e., de-signing and making the product lines or assortments that the companyoffers to its clients), here generically denoted by P1, . . ., Pj, . . ., and Pn.
The company’s plants possess several resources, R1, . . ., Ri, . . ., andRm, to make the different assortments. The quantity available of eachresource is, b1, . . ., bi, . . ., and bm, respectively. The allocation of resourcesto the production is as follows: allocate aij units of the resource Ri to makeone unit of the assortment Pj.
The market absorbs all the production and the revenue per unit ofproduct sold is cj Euros. The Company’s CEO needs to know the next yearproduction plan (activity levels) in such a way that the resource constraintsshould be taken into account and the overall revenue to be earned from theselling of the assortments in the market should be maximized.
Example 1 (A Classical Product Mix Resource Allocation Problem).
J.R. Figueira (IST) FIO March 28-29, 2016 24 / 45
3. The optimal primal simplex Tableau
Optimal Primal Simplex Tableau
((i) ∈ B) ((i) ∈ B) cj c1 . . . cj . . . cn 0 . . . 0 . . . 0 Sol.
c(i) x(i) xj x1 . . . xj . . . xn xn+1 . . . xn+i . . . xn+m bi
c(1) x(1) a11 . . . a1j . . . a1n a1,n+1 . . . a1,n+i . . . a1,n+m b1...
......
. . ....
. . ....
.... . .
.... . .
......
c(i) x(i) ai1 . . . aij . . . ain ai,n+1 . . . ai,n+i . . . ai,n+m bi...
......
. . ....
. . ....
.... . .
.... . .
......
c(m) x(m) am1 . . . amj . . . amn am,n+1 . . . am,n+i . . . am,n+m bm
zj z1 . . . zj . . . zn zn+1 . . . zn+i . . . zn+m
1 z cj = (zj − cj) (z1 − c1) . . . (zj − cj) . . . (zn − cn) (zn+1 − cn+1) . . . (zn+i − cn+i) . . . (zn+m − cn+m)m
∑i=1
c(i)b(i)
I Primal optimal solution (for the decision and slack variables):x?(i) = bi, i = 1, . . . , m.
z? =m
∑i=1
c(i)bi.
I But, this Tableau provides more information than merely the values of the primal variables.
J.R. Figueira (IST) FIO March 28-29, 2016 25 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
3. The optimal primal simplex Tableau
Some Questions
I For the assortments that are not produced (the ones corresponding to thenon-basic decision variables) how much more expensive should we make theirprices in order to produce them in the future?
I What is the value of an extra unit of a scarce resource?
Simplex provide us the economic information for the answers.
I As for the decision variables the reduced costs give us the necessary priceincreasing.
I As for the value of resources, shadow prices can be seen as the marginaleffect of a decrease or an increase of resources availabilities.
J.R. Figueira (IST) FIO March 28-29, 2016 26 / 45
4. What are the shadow prices?
An Important Definition
Associated with an optimal solution are shadow prices (also referred to asdual variables, marginal values, marginal valuations, . . .) for the constraints.The shadow price associated with a particular constraint corresponds to themodification in the optimal value of the objective function per unit increasein the right-hand-side (RHS) value for that constraint.
Shadow prices for abundant resources are equal to zero. The companydoes not give valuation to its abundant resources.
Definition 1 (Shadow Prices on the Constraints).
J.R. Figueira (IST) FIO March 28-29, 2016 27 / 45
5. Another view of the problem
A Related Minimization Problem
The CorkMaxPro Company’s accountant is confronted with the assess-ment of the value of the company since there is a strong possibility ofrenting it to a Spanish investor for one year. The accountant should reportthe value to the CEO, who keeps in mind that renting the resources mustbe at least as favorable as using the capacity itself (i.e., the rent should beat least equal to the earnings obtained from producing the assortments).
The company’s plants have bi available units of resource Ri, for i =1, . . . , m. What should be the rent per unit of resource? And what shouldbe the minimum value of the overall rent in such a way that the companywill know what minimum offer will be economically acceptable for renting?
Producing one unit of Pj needs a1j units of resource R1, a2j units ofresource R2, . . ., and amj units of resource Rm. The overall rent for suchquantities of resources should be greater than, or equal to, the revenueearned from one unit of the assortment Pj, cj, for j = 1, . . . , n.
Example 2 (Renting the CorkMaxPro Company).
J.R. Figueira (IST) FIO March 28-29, 2016 28 / 45
5. Another view of the problem
A Related Minimization Problem
The CorkMaxPro Company’s accountant is confronted with the assess-ment of the value of the company since there is a strong possibility ofrenting it to a Spanish investor for one year. The accountant should reportthe value to the CEO, who keeps in mind that renting the resources mustbe at least as favorable as using the capacity itself (i.e., the rent should beat least equal to the earnings obtained from producing the assortments).
The company’s plants have bi available units of resource Ri, for i =1, . . . , m. What should be the rent per unit of resource? And what shouldbe the minimum value of the overall rent in such a way that the companywill know what minimum offer will be economically acceptable for renting?
Producing one unit of Pj needs a1j units of resource R1, a2j units ofresource R2, . . ., and amj units of resource Rm. The overall rent for suchquantities of resources should be greater than, or equal to, the revenueearned from one unit of the assortment Pj, cj, for j = 1, . . . , n.
Example 2 (Renting the CorkMaxPro Company).
J.R. Figueira (IST) FIO March 28-29, 2016 28 / 45
5. Another view of the problem
A Related Minimization Problem
The CorkMaxPro Company’s accountant is confronted with the assess-ment of the value of the company since there is a strong possibility ofrenting it to a Spanish investor for one year. The accountant should reportthe value to the CEO, who keeps in mind that renting the resources mustbe at least as favorable as using the capacity itself (i.e., the rent should beat least equal to the earnings obtained from producing the assortments).
The company’s plants have bi available units of resource Ri, for i =1, . . . , m. What should be the rent per unit of resource? And what shouldbe the minimum value of the overall rent in such a way that the companywill know what minimum offer will be economically acceptable for renting?
Producing one unit of Pj needs a1j units of resource R1, a2j units ofresource R2, . . ., and amj units of resource Rm. The overall rent for suchquantities of resources should be greater than, or equal to, the revenueearned from one unit of the assortment Pj, cj, for j = 1, . . . , n.
Example 2 (Renting the CorkMaxPro Company).
J.R. Figueira (IST) FIO March 28-29, 2016 28 / 45
5. Another view of the problem
A Related Minimization Problem
The CorkMaxPro Company’s accountant is confronted with the assess-ment of the value of the company since there is a strong possibility ofrenting it to a Spanish investor for one year. The accountant should reportthe value to the CEO, who keeps in mind that renting the resources mustbe at least as favorable as using the capacity itself (i.e., the rent should beat least equal to the earnings obtained from producing the assortments).
The company’s plants have bi available units of resource Ri, for i =1, . . . , m. What should be the rent per unit of resource? And what shouldbe the minimum value of the overall rent in such a way that the companywill know what minimum offer will be economically acceptable for renting?
Producing one unit of Pj needs a1j units of resource R1, a2j units ofresource R2, . . ., and amj units of resource Rm. The overall rent for suchquantities of resources should be greater than, or equal to, the revenueearned from one unit of the assortment Pj, cj, for j = 1, . . . , n.
Example 2 (Renting the CorkMaxPro Company).
J.R. Figueira (IST) FIO March 28-29, 2016 28 / 45
5. Another view of the problem
The Linear Programming (LP) Dual Model
minimize w(y) = b1y1 + . . . + biyi + . . . + bmym
subject to: a11y1 + . . . + ai1yi + . . . + am1ym > c1...
......
a1jy1 + . . . + aijyi + . . . + amjym > cj...
......
a1my1 + . . . + aimyi + . . . + amnym > cn
y1, . . . yi, . . . ym > 0
J.R. Figueira (IST) FIO March 28-29, 2016 29 / 45
6. More on the economic interpretation
Some Remarks
I The overall valuation provided by the accountant to the CEO is equal to itsoptimal production plan of the original problem. It seems quite natural andthe result comes from the Duality Theorems (this results is always true).
I Abundant resources have a zero valuation. Again, it seems quite naturalbecause we do not use all the availability of resources. It is also a result fromDuality (Slackness Complementarity): If a constraint is non-biding in theoptimal primal Tableau its dual value, in the optimal dual solution, is zero.
I When, for a given product, the imputed costs are higher than its per unitrevenue, the company does not make its production. This is another resultfrom the Duality (Slackness Complementarity). If a constraint for the dualproblem is non-biding, then the associated primal decision variable has a zerovalue. From an economic point of view it means that non-biding constraintshave zero valuations.
J.R. Figueira (IST) FIO March 28-29, 2016 30 / 45
6. More on the economic interpretation
Some Remarks
I The overall valuation provided by the accountant to the CEO is equal to itsoptimal production plan of the original problem. It seems quite natural andthe result comes from the Duality Theorems (this results is always true).
I Abundant resources have a zero valuation. Again, it seems quite naturalbecause we do not use all the availability of resources. It is also a result fromDuality (Slackness Complementarity): If a constraint is non-biding in theoptimal primal Tableau its dual value, in the optimal dual solution, is zero.
I When, for a given product, the imputed costs are higher than its per unitrevenue, the company does not make its production. This is another resultfrom the Duality (Slackness Complementarity). If a constraint for the dualproblem is non-biding, then the associated primal decision variable has a zerovalue. From an economic point of view it means that non-biding constraintshave zero valuations.
J.R. Figueira (IST) FIO March 28-29, 2016 30 / 45
6. More on the economic interpretation
Some Remarks
I The overall valuation provided by the accountant to the CEO is equal to itsoptimal production plan of the original problem. It seems quite natural andthe result comes from the Duality Theorems (this results is always true).
I Abundant resources have a zero valuation. Again, it seems quite naturalbecause we do not use all the availability of resources. It is also a result fromDuality (Slackness Complementarity): If a constraint is non-biding in theoptimal primal Tableau its dual value, in the optimal dual solution, is zero.
I When, for a given product, the imputed costs are higher than its per unitrevenue, the company does not make its production. This is another resultfrom the Duality (Slackness Complementarity). If a constraint for the dualproblem is non-biding, then the associated primal decision variable has a zerovalue. From an economic point of view it means that non-biding constraintshave zero valuations.
J.R. Figueira (IST) FIO March 28-29, 2016 30 / 45
6. More on the economic interpretation
Some Remarks
I The overall valuation provided by the accountant to the CEO is equal to itsoptimal production plan of the original problem. It seems quite natural andthe result comes from the Duality Theorems (this results is always true).
I Abundant resources have a zero valuation. Again, it seems quite naturalbecause we do not use all the availability of resources. It is also a result fromDuality (Slackness Complementarity): If a constraint is non-biding in theoptimal primal Tableau its dual value, in the optimal dual solution, is zero.
I When, for a given product, the imputed costs are higher than its per unitrevenue, the company does not make its production. This is another resultfrom the Duality (Slackness Complementarity). If a constraint for the dualproblem is non-biding, then the associated primal decision variable has a zerovalue. From an economic point of view it means that non-biding constraintshave zero valuations.
J.R. Figueira (IST) FIO March 28-29, 2016 30 / 45
6. More on the economic interpretation
Some Remarks
I The overall valuation provided by the accountant to the CEO is equal to itsoptimal production plan of the original problem. It seems quite natural andthe result comes from the Duality Theorems (this results is always true).
I Abundant resources have a zero valuation. Again, it seems quite naturalbecause we do not use all the availability of resources. It is also a result fromDuality (Slackness Complementarity): If a constraint is non-biding in theoptimal primal Tableau its dual value, in the optimal dual solution, is zero.
I When, for a given product, the imputed costs are higher than its per unitrevenue, the company does not make its production. This is another resultfrom the Duality (Slackness Complementarity). If a constraint for the dualproblem is non-biding, then the associated primal decision variable has a zerovalue. From an economic point of view it means that non-biding constraintshave zero valuations.
J.R. Figueira (IST) FIO March 28-29, 2016 30 / 45
6. More on the economic interpretation
Shadow Prices as Opportunity Costs
I An increasing in the availability of a scarce resource results in an opportunityto make more revenue.
I Analogously, a decreasing in the availability of a scarce resource results in aloose opportunity to make more revenue.
I The shadow price represents a cost of the lost opportunity.
J.R. Figueira (IST) FIO March 28-29, 2016 31 / 45
7. Lagrangian Duality
Lagrangian Duality (1)
I Define the Lagrangian L that corresponds to the primal problem (P).
L(x, y) = c>x + y>(Ax− b).
I The vector y plays the role of a vector of Lagrange multipliers for theconstraints in the primal problem (P).
I The following definition is a fundamental one in unconstrained optimization.
The point (x∗, y∗) is a saddle point of the Lagrangian L if and only if
1. x∗ ≥ 0, y∗ ≥ 0 and
2. L(x, y∗) 6 L(x∗, y∗) 6 L(x∗, y), for all x ≥ 0, y ≥ 0.
Definition 2 (Saddle Point).
J.R. Figueira (IST) FIO March 28-29, 2016 32 / 45
7. Lagrangian Duality
Lagrangian Duality (1)
I Define the Lagrangian L that corresponds to the primal problem (P).
L(x, y) = c>x + y>(Ax− b).
I The vector y plays the role of a vector of Lagrange multipliers for theconstraints in the primal problem (P).
I The following definition is a fundamental one in unconstrained optimization.
The point (x∗, y∗) is a saddle point of the Lagrangian L if and only if
1. x∗ ≥ 0, y∗ ≥ 0 and
2. L(x, y∗) 6 L(x∗, y∗) 6 L(x∗, y), for all x ≥ 0, y ≥ 0.
Definition 2 (Saddle Point).
J.R. Figueira (IST) FIO March 28-29, 2016 32 / 45
7. Lagrangian Duality
Lagrangian Duality (1)
I Define the Lagrangian L that corresponds to the primal problem (P).
L(x, y) = c>x + y>(Ax− b).
I The vector y plays the role of a vector of Lagrange multipliers for theconstraints in the primal problem (P).
I The following definition is a fundamental one in unconstrained optimization.
The point (x∗, y∗) is a saddle point of the Lagrangian L if and only if
1. x∗ ≥ 0, y∗ ≥ 0 and
2. L(x, y∗) 6 L(x∗, y∗) 6 L(x∗, y), for all x ≥ 0, y ≥ 0.
Definition 2 (Saddle Point).
J.R. Figueira (IST) FIO March 28-29, 2016 32 / 45
7. Lagrangian Duality
Lagrangian Duality (1)
I Define the Lagrangian L that corresponds to the primal problem (P).
L(x, y) = c>x + y>(Ax− b).
I The vector y plays the role of a vector of Lagrange multipliers for theconstraints in the primal problem (P).
I The following definition is a fundamental one in unconstrained optimization.
The point (x∗, y∗) is a saddle point of the Lagrangian L if and only if
1. x∗ ≥ 0, y∗ ≥ 0 and
2. L(x, y∗) 6 L(x∗, y∗) 6 L(x∗, y), for all x ≥ 0, y ≥ 0.
Definition 2 (Saddle Point).
J.R. Figueira (IST) FIO March 28-29, 2016 32 / 45
7. Lagrangian Duality
Lagrangian Duality (1)
I Define the Lagrangian L that corresponds to the primal problem (P).
L(x, y) = c>x + y>(Ax− b).
I The vector y plays the role of a vector of Lagrange multipliers for theconstraints in the primal problem (P).
I The following definition is a fundamental one in unconstrained optimization.
The point (x∗, y∗) is a saddle point of the Lagrangian L if and only if
1. x∗ ≥ 0, y∗ ≥ 0 and
2. L(x, y∗) 6 L(x∗, y∗) 6 L(x∗, y), for all x ≥ 0, y ≥ 0.
Definition 2 (Saddle Point).
J.R. Figueira (IST) FIO March 28-29, 2016 32 / 45
7. Lagrangian Duality
Lagrangian Duality (2)
I The function L(x, y∗) attains a maximum with respect to x at x = x∗ overall x ≥ 0.
I Conversely, L(x∗, y) attains a minimum with respect to y at y = y∗ over ally ≥ 0.
The saddle point conditions of Definition 2 are necessary and sufficient forx∗ to solve (P) and for y∗ to solve (D).
Theorem 5 (Optimality Conditions).
J.R. Figueira (IST) FIO March 28-29, 2016 33 / 45
7. Lagrangian Duality
Lagrangian Duality (2)
I The function L(x, y∗) attains a maximum with respect to x at x = x∗ overall x ≥ 0.
I Conversely, L(x∗, y) attains a minimum with respect to y at y = y∗ over ally ≥ 0.
The saddle point conditions of Definition 2 are necessary and sufficient forx∗ to solve (P) and for y∗ to solve (D).
Theorem 5 (Optimality Conditions).
J.R. Figueira (IST) FIO March 28-29, 2016 33 / 45
7. Lagrangian Duality
Lagrangian Duality (2)
I The function L(x, y∗) attains a maximum with respect to x at x = x∗ overall x ≥ 0.
I Conversely, L(x∗, y) attains a minimum with respect to y at y = y∗ over ally ≥ 0.
The saddle point conditions of Definition 2 are necessary and sufficient forx∗ to solve (P) and for y∗ to solve (D).
Theorem 5 (Optimality Conditions).
J.R. Figueira (IST) FIO March 28-29, 2016 33 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (1)
I Consider our problem (P) as follows, in the equality form,
maxx
{c>x : (Ax− b) = 0, x ≥ 0
}.
I It implies,
g(y) = maxx
{c>x + y>(Ax− b) : x ≥ 0, y ∈ Rm
}.
I For any y ∈ Rm, g(y) provides an upper bound for (P).
I To get the best upper bound, we should
miny
g(y).
I This problem is called the Lagrangian dual and g(y) the dual function.
J.R. Figueira (IST) FIO March 28-29, 2016 34 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
7. Lagrangian Duality
Building the Dual (2)
I Let us analyze the function g(y).
g(y) = maxx
{c>x + y>(Ax− b)
}= y>b + max
x
{(y>A− c>)x
}.
I It should be noticed that,
g(y) =
+∞ if (y>A− c>)j < 0, for at least one j
y>b if (y>A− c>)j > 0, for all j
I Since +∞ is not an interesting upper bound, we should focus our attentionfor y ∈ Rm such that y>A− c> ≥ 0. The dual (D) can thus be written asfollows.
miny
{y>b : y>A ≥ c, y ∈ Rm
}.
I The only difference is that we started with Ax = b and will have y with freecomponents.
J.R. Figueira (IST) FIO March 28-29, 2016 35 / 45
Part
More on Post-Optimality
Contents
Adding a New Constraint
Add the constraint x1 + 2x2 6 8 to:
maximize x1 + 2x2subject to: x1 + x2 6 6
x2 6 3
with x1, x2 > 0.
x1
x2
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
The previous solution is no more optimal:
cj 1 2 0 0Basis xj x1 x2 x3 x4 bi
1 x1 1 0 1 −1 32 x2 0 1 0 1 3
zj 1 2 1 1cj 0 0 1 1 9
I How to deal with this case?
I Add one line to the tableau.
(related to the constraint)
I This implies to add a column.
(related to the new slack variable, x5)
I Put x5 as a function of x3, x4.
(the two non-basic variables)
J.R. Figueira (IST) FIO March 28-29, 2016 38 / 45
Adding a New Constraint
I The new constraint: x1 + 2x2 + x5 = 8.
I Equivalently: x5 = 8− x1 − 2x2.
(where x5 is the slack variable)
I We know that: x1 = 3− x3 + x4.
(from the first row of the tableau)
I We also know that: x2 = 3− x4.
(from the second row of the tableau)
I We have:
x5 = 8− (3− x3 + x4)− 2(3− x4).
x5 = −1 + x3 + x4
−x3 − x4 + x5 = −1I Fill the yellow zones!
I Observe that the solution is optimal but notfeasible!
I It‘s time to use Dual Simplex Algorithm.
I x5 leaves and x4 enters.
The previous solution is no more optimal:
cj 1 2 0 0 0Basis xj x1 x2 x3 x4 x5 bi
1 x1 1 0 1 −1 0 32 x2 0 1 0 1 0 30 x5 0 0 −1
�� ��−1 1 −1zj 1 2 1 1 0cj 0 0 1 1 0 9
cj 1 2 0 0 0Basis xj x1 x2 x3 x4 x5 bi
1 x1 1 0 2 0 −1 42 x2 0 1 −1 0 1 20 x4 0 0 1 1 −1 1
zj 1 2 0 0 1cj 0 0 0 0 1 8
Multiple Optima!
J.R. Figueira (IST) FIO March 28-29, 2016 39 / 45
Initial Dual Simplex Tableau (Iteration 0)
(q ∈ B) (q ∈ B) cj c1 . . . ck . . . cn 0 . . . 0 . . . 0 Solution
cq xq xj x1 . . . xk . . . xn xn+1 . . . xn+` . . . xn+m bi0 xn+1 −a11 . . . −a1k . . . −a1n 1 . . . 0 . . . 0 −b1...
......
. . ....
. . ....
.... . .
.... . .
......
0 xn+` −a`1 . . .�� ��−a`k . . . −a`n 0 . . . 1 . . . 0 −b`
......
.... . .
.... . .
......
. . ....
. . ....
...0 xn+m −am1 . . . −amk . . . −amn 0 . . . 0 . . . 1 −bm
zj z1 . . . zk . . . zn 0 . . . 0 . . . 01 z zj − cj z1 − c1 . . . zk − ck . . . zn − cn 0 . . . 0 . . . 0 0
minimize 6x1 + 3x2subject to: x1 > 1
x1 + x2 > 2
with x1, x2 > 0.
minimize 6x1 + 3x2subject to: −x1 + x3 = −1
−x1 + −x2 + x4 = −2
with x1, x2, x3, x4 > 0.
J.R. Figueira (IST) FIO March 28-29, 2016 40 / 45
Dual Simplex Algorithm: minx{c>x : −Ax ≤ −b, x ≥ 0}
Algorithm 1 Dual Simplex
Input: A ∈ Rm×n, b ∈ Rm, c ∈ Rn.Output: x∗ ∈ Rn
+ such that Ax ≥ b.
1: x∗i ← −bi, i ∈ B = {n + 1, . . . , n + m}, and x∗j ← 0, j ∈ N = {1, . . . , n};2: compute cj, j ∈ N (cj ← −cj);
3: while (∀t ∈ {i : bi < 0, i = 1, . . . , m} : {j : −atj < 0, j ∈ N} 6= {}) do
4: select ` ∈ {i : bi < 0, i = 1, . . . , m} and the corresponding x(`);5: select k ∈ arg min
j∈N
{cj/− aj` : −aj` < 0
}and the corresponding xk;
6: pivoting on −a`k;7: x∗(i) ← bi, i = 1, . . . , m, and x∗j ← 0, j ∈ N ;
8: compute cj, j ∈ N ;9: end while
10: if (∃t ∈ {i : bi < 0, i = 1, . . . , m}) and ({j : −atj < 0, j ∈ N} = {}) then11: x∗ ← −∞ (primal infeasibility);12: end if
J.R. Figueira (IST) FIO March 28-29, 2016 41 / 45
A Second Small Example
minimize 6x1 + 3x2subject to: x1 > 1
x1 + x2 > 2
with x1, x2 > 0.
x1
x2
0 1 2 3 40
1
2
3
4
cj 6 3 0 0Basis xj x1 x2 x3 x4 bi
0 x3 −1 0 1 0 −10 x4 −1
�� ��−1 0 1 −2zj 0 0 0 0cj −6 −3 0 0 0
0 x3
�� ��−1 0 1 0 −13 x2 1 1 0 −1 2
zj 3 3 0 −3cj −3 0 0 −3 6
6 x1 1 0 −1 0 13 x2 0 1 1 −1 1
zj 6 3 −3 −3cj 0 0 −3 −3 9
J.R. Figueira (IST) FIO March 28-29, 2016 42 / 45
Some questions
Questions:
1. Consider the optimal tableau of the previous example.
2. Identify the solution of the dual problem.
3. Build the dual of the primal.
4. Show the optimality of solutions through the complementary slacknessproperty.
J.R. Figueira (IST) FIO March 28-29, 2016 43 / 45
Part
Bibliography
References
Bazaraa, M., Jarvis, J., and Sherali, H. (1990).
Linear Programming and Network Flows.John Wiley & Sons, New York, USA, second edition.
Goldfarb, D. and Todd, M. J. (1989).
Linear programming.In Nemhauser, G. L., Rinnoy Kan, A. H. G., and Todd, M. J., editors, Optimization, volume 1 of Handbooks in Operations Research andManagement Science, pages 141–170. North Holland, Amsterdam, The Netherlands.
Hillier, F. and Lieberman, G. (2005).
Introduction to Operations Research.The McGraw-Hill Companies, Inc., New York, USA, eighth edition.
Taha, H. (2010).
Operations Research: An Introduction.Prentice-Hall, Upper Saddle River, New Jersey, USA, 9th edition.
Williams, P. (1993).
Model Solving in Mathematical Programming.John Wiley & Sons, Chichester, UK.
Williams, P. (1999).
Model Building in Mathematical Programming.John Wiley & Sons, Chichester, UK.