9/4/19

1

AM/ES121:IntrotoOptimizationModelsandMethods

Fall2019

YilingChenSEAS

Lecture1:CourseOverview

LessonPlan•  Mathematicalprogramming•  Reviewsyllabus,schedule

–  Coursewebsite:am121.seas.harvard.edu–  PiazzaforQ&A– GradeScopeforsubmittingpsets

•  Overviewoftopics(withmotivatingexamples)–  LinearProgramming–  IntegerProgramming–  StochasticProgramming

2

9/4/19

2

MathematicalProgramming

•  Setofmethodstomodelandsolveoptimizationproblemswithcontinuousorintegervariables

•  Programming≠computerprogramming•  Programming=Planning

3

MathematicalProgramming

•  Weemphasizemodeling•  Methodswithelegantmathematics•  Applications

4

Real-worldproblems

…LP IP MDP

Simplex Branch-and-Bound

Model

Method

9/4/19

3

ProminentApplications

•  OptimalcrewschedulingsavesAmericanairlines>20milliondollarsperyear

•  OptimizationandanalyticstoolssaveUPS85millionmileseachyear,savingaround$125m(howmanyroutescanadriverwith25packageschoosefrom?)(and…)

•  Amazonwouldn’texistwithoutoptimization•  GooglePageRank–atoolforindexingtheWeb–isa

calculationonaMarkovchain.•  ClearingNationwideKidneyExchanges,savinglives.

5

Campaignstops:Theproblemofthetravellingpolitician

ByWILLIAMCOOK(GeorgiaTech)

•  What'sthequickestwaytovisitall99countiesinIowa?

•  Republicanprimary,2011-12.PresidentialcaucusinIowa.

•  Swingstate(butDemocraticsince2008).

6

9/4/19

4

Solution(thefastestpolitician!)

7

TheShortestPossibleBaseballRoadtrip

•  BenBlatt(classof2013)

–  http://www.wsj.com/articles/SB10001424052702303657404576357560436672964

–  https://harvardsportsanalysis.wordpress.com/2011/06/03/roadtrip/

8

Ben,asportsanalyticswizard,lovesbaseball.Eric,hisbestfriend,hatesit.ButwhenBenwritesanalgorithmfortheoptimalbaseballroadtrip—animpossibledreamofseeingeverypitchof30gamesin30stadiumsin30days—whowillhecallontotakeshiftsbehindthewheel,especiallywhenthoseshiftsincludenineteenhoursstraightfromPhoenixtoKansasCity?Eric,ofcourse.WillEricregretit?Mostdefinitely.

9/4/19

5

Syllabus•  Prerequisite:AM21borMath21b(linearalgebra),basicknowledgeofprobabilities

•  RecommendedText:“OperationsResearch:ModelsandMethods”,byJensenandBard.

•  Optionaltext:“AMPL”,byFourer,GayandKernighan

9

•  7problemsets(mixtureofmodeling,computation,andtheory).EncourageuseofMatlab/Mathematica– 4latedays,atmost1latedayperassignment

•  Two“Extremeoptimization”teamassignments– what’sthat?(Extremevalues:communication,simplicity,feedback,courage,respect).

•  Twoin-classmidterms;Nofinal.•  Gradebreakdownis30%problemsets,20%extremeoptimization,50%midterms

10

Syllabus

9/4/19

6

•  We’llbeusingAMPLandCPLEX

•  AMPLisamodelingtool.Literally“AMathematicalProgrammingLanguage”(AMPL)

•  VerysimplewaytowritedownIPsandLPs•  Setthisupandtestthisweek!

•  CPLEXisasolver.WasdevelopedbyILOG,nowownedbyIBM.Industrialstrength.

11

Syllabus

•  Collaborationpolicy– Cancollaborateinplanningandthinkingthroughsolutions,butmustwriteupyourownsolutionswithoutcheckingthisoverwithanotherstudent.

– Donotpasssolutionstoproblemsetsnoracceptthemfromanotherstudent.(It’sNOTOKtogetsolutionstopreviousyears’problemsets.)

•  Engineeringstudents,particularlyS.B.concentrators,shouldenrollunderES121.Thiswillensurethecoursecountsasanelective.

12

Syllabus

9/4/19

7

GetReadyfortheCourse•  Bookmarkthecoursewebsitehttp://am121.seas.harvard.edu/

•  CompleteAssignment0byMonday9/9–  Linearalgebrareview–  SignupforPiazza(bookmarkit!)–  InstallAMPL

•  Checkoutthe“Courselogisticsoverview”documentontheResourcespageofthecoursewebsite.

•  Myofficehoursthisweek:Wednesday1:30-2:30,Friday9:15-10:00,MD233

•  Sectionsstartnextweek.Attendsections!•  Piazza(allQ&A)•  GradeScope(assignmentsubmissions)

13

RoughSchedule

•  6weeksonlinearprogramming•  4weeksonintegerprogramming•  2weeksstochasticprogramming•  Combinedwithapplications:

– 2extremeoptimizationprojects

•  2breakoutmodelingsessions

14

9/4/19

8

Optimization

•  Theproblemofmakingdecisionstomaximizeorminimizeanobjective,maybeinthepresenceofcomplicatingconstraints– Decisionvariables– Objectivefunction– Constraints

15

ASimpleOptimizationProblem

•  minimizey=(x-2)2+3

Thisisnon-linear.Inthiscourse,wefocusonlinearmodels. 16

minimizey=(x-2)2+3s.t.x≥3

9/4/19

9

ALinearProgram

17

Howwouldyouestablishtheoptimalityofthisalgebraically? 18

x1=3.6,x2=1.4,value320

Solving(graphically!)

5 ⇤ (1) : 35x1 + 10x2 � 140

7.5 ⇤ (2) : 15x1 + 90x2 � 180

+ : 50x1 + 100x2 � 320

9/4/19

10

ADistributionNetwork

19

Shipgoodsfromfactoriestowarehousestominimizetotalcostwhilemeetingdemandsandcapacityconstraints.

ADistributionNetwork

•  Decisionvars.:amounttoshiponeachlane,xF1-F2,xF1-DC,xF1-W1,xF2-DC,xDC-W2,xW1-W2,xW2-W1.

•  Objective:minimizetotalshippingcost•  Constraints:

– amountshippedin–amountshippedout=requiredamount

– amountonalane<=capacity– amountonalane>=0

20

9/4/19

11

OptimalAnimalFeed!•  Producethefeedmixthatsatisfiesnutritionrequirements

withminimalcost•  Requirements:

–  Calcium:atleast0.8%butnotmorethan1.2%–  Protein:atleast22%–  Fiber:atmost5%

•  NutrientContentsandCostsofIngredients

Ingredient Calcium Protein Fiber Unitcost(cents/kg)

Limestone 0.38 0.0 0.0 10.0

Corn 0.001 0.09 0.02 30.5

Soybeanmeal

0.002 0.5 0.08 90.0

21

Whatwe’llcoverforLinearProgramming

•  Modeling:Givenaproblem,formulatethelinearprogrammingmathematicalmodel.

•  SolvetheformulatedLPs– Simplexmethod– Sensitivityanalysis– Duality

22

9/4/19

12

IntegerProgramming(IP)

•  Decisionvariablescanonlytakeintegervalues

•  Linearprogrammingmodel+integralityconstraints

•  Example:

23

max5x1+8x2s.t.x1+x2≤65x1+9x2≤45x1,x2≥0,integer

also“mixedintegerprograms”with{0,1},integerandcontinuous

LinearProgramsvs.IntegerPrograms

•  LinearPrograms– Continuousdecisionvariables

•  IntegerPrograms– Discretedecisionvariables,musttakeintegralvalues– Smallerfeasiblespace– Generallyhardertosolve!

•  ThenaturalideaofroundingLPsolutionforIPdoesnotwork...

24

9/4/19

13

LinearProgramsvs.IntegerPrograms

•  LinearPrograms– Continuousdecisionvariables

•  IntegerPrograms– Discretedecisionvariables,musttakeintegralvalues– Smallerfeasiblespace– Generallyhardertosolve!

•  ThenaturalideaofroundingLPsolutionforIPdoesnotwork...

25

0

1

2

3

4

5

0 1 2 3 4 5

(376/193,950/193) LP solution

(5,0) IP solution

9/4/19

14

TakingthefunoutofSudoku

27

•  Decisionvariables–  :equals1ifcellatrowiandcolumnjhasvaluev,otherwise0.

•  Constraints:–  Eachrowmusthaveone1…9–  Eachcolumnmusthaveone1…9–  Each3-by-3sectionmusthaveone1…9–  Eachcellmusthaveavaluefrom1…9

•  Objective– Anything(it’sa“constraintsatisfactionproblem”,butcanbesolvedviaanIPbyadoptinganyobjective.)

TakingthefunoutofSudoku

28

bvi,j

9/4/19

15

TheAssignmentProblem

•  Therearepeopleavailabletocarryoutjobs

•  Eachpersonhastobeassignedtoexactlyonejob

•  Person ifassignedtojob incurscost •  Goal:findaminimumcostassignment

29

n n

i j cij

Whatwe’llcoverforIntegerProgramming

•  Modeling:Givenaproblem,formulatetheintegerprogrammingmathematicalmodel.

•  SolvetheformulatedIPs– Branch-and-Bound– Cuttingplane– Branch-and-cut

30

9/4/19

16

IntroducingUncertainty

•  Sofar,we’vebeenthinkingaboutusingmath.programmingtosolveproblemswithknownmodelparameters.

•  Butmanyreal-worldproblemsinvolveuncertainty.

•  Stochasticprogrammingisanapproachforsolvingoptimizationproblemsthatinvolveuncertainty.

31

Whatwe’llcoverforStochasticProgramming

•  Markovchains(tomodelanunderlyingstochasticprocess)

•  MarkovDecisionProcesses(MDPs),whichwe’llsolvevialinearprogramming

•  Scalingupviatwo-stage,stochasticoptimization

32

9/4/19

17

MarkovChains•  Astochasticprocess(asequenceofrandomvariables)thathastheMarkovproperty

•  Markovproperty:stateattimet+1onlydependsonstateattimet(historybeforetdoesnotmatter)

33

MarkovChains•  Astochasticprocess(asequenceofrandomvariables)thathastheMarkovproperty

•  Markovproperty:stateattimet+1onlydependsonstateattimet(historybeforetdoesnotmatter)

•  Eg.Machinecondition

34

State Condition

0 Goodasnew

1 Minordeterioration

2 Majordeterioration

3 Inoperable

State 0 1 2 3

0 0 7/8 1/16 1/16

1 0 3/4 1/8 1/8

2 0 0 1/2 1/2

3 0 0 0 1

9/4/19

18

MarkovDecisionProcesses(MDP)

•  RatherthanjustobservingtheMarkovchain,wecanpickanactiontotakeineachstate.

•  Givenan(action,state)pair,wetransitionintothenextstate.

•  Eachstateisassociatedwithacost(orvalue).

•  TheproblemofchoosinganactionineachstateisreferredasaMarkovDecisionProcess.

35

Example:MachineMaintenance

36

State Condition Cost

0 Goodasnew 0

1 Minordeterioration 1000

2 Majordeterioration 2000

3 Inoperable 6000

Decision Action RelevantStates

1 Donothing 0,1,2

2 Overhaul(returnthesystemtostate1) 2

3 Replace(returnthesystemtostate0) 1,2,3

Goal:findamaintenanceplanthatminimizestotalexpectedcost

9/4/19

19

AM121:LearningOutcomes•  Beabletotransformarealworldproblemintoamathematicalprogrammingmodel

•  Masterthetechniquesusedtosolvethemathematicalprogrammingmodel

•  Beabletosolvetheproblemusingcommercialsoftwarepackages,i.e.CPLEX/AMPL

•  Feelempowered!•  Comeseemeduringmyofficehoursifyouhavequestionsaboutthecourse.Assignmentzeroposted!

37