L15 linear programming - University Of Marylandtomg/course/cmsc764/L15... · 1948: Soviets blockade...

LINEAR PROGRAMMING

1

THE ORIGINAL “BIG DATA”

•One week later : airlift begins •Big operation: $250 million•3 Billion today in 2 weeks

•3 routes•5 nations•Plane landed every 30 seconds•2,346,000 tons of freight

Difficult scheduling problem!

1948: Soviets blockade West Berlin

2

THE ORIGINAL “BIG DATA”

•George B. Dantzig (this guy) appointed to solve problem•LP formulation•Simplex method

•US Air Force implementation•IBM calculation machines•Conclusion: Intractable

50 unknownsSource: Klein, 20133

SIMPLEX

Is it convex? Why?

{x : Ax b}

4

SIMPLEX METHOD

Ax b

x0

How could you solve this?

Solution is always on vertex, why?

maximize cTx

subject to Ax b

c5

SIMPLEX METHOD

x0

Ax b

How could you solve this?

Solution is always on vertex, why?

Gradient descent!

maximize cTx

subject to Ax b

c6

SIMPLEX METHODmaximize cTx

subject to Ax b

c

7

YOU JUST WON A NOBEL PRIZE!

(if you lived 60 years ago)

Dantzig wins National Medal of Science

Leonid Kantorovich, Tjalling Koopmanswin Nobel Prize

8

THE BIG IDEA

Q: How many constraints are activeat each vertex?

maximize cTx

subject to Ax b

9

THE BIG IDEA

Q: How many constraints are activeat each vertex?

A: N

• Pick N equations, solve them• Drop an equation, swap in a new one• Solve new system• Try to decrease energy

maximize cTx

subject to Ax b

10

SETUP

Simplex Tableau

represent constraints in matrix form

0

@1 �c1 �c2 �c3 �c4 �c5 00 a11 a12 a13 a14 a15 b10 a21 a22 a23 a24 a25 b2

1

A

maximize cTx

subject to Ax = b

x � 0

maximize t

subject to Ax = b

t = cTx

x � 0

11

STEP 1: CHOOSE BASIC SOLUTION

Basic Solution: A set of N variables that are feasible: they satisfy both equality and inequality constraints

x1 x2

All non-basic variables are assumed to be zero in candidate solution

0


1

A

12

STEP 1: CHOOSE BASIC SOLUTION

eliminate

x1 = b1

x2 = b2

0


1

A

0

BBBBBB@

b0b1b2000

1

CCCCCCA

<latexit sha1_base64="VV6vpTshQzvYoNKrINJQkVH/xXY=">AAAIPXicfVXdjhs1FJ4WaEr427aX3IxIixZYVplQiV62XYTgYkVB7Hal9TbyOJ6MiX+mtqeb1PUb8DTcgsRz8ADcIW65hONJQhPPZEcaz7G/z9/xOcf25BVnxg6Hf1y7/sabb93o3Xy7/867773/wd6t26dG1ZrQE6K40mc5NpQzSU8ss5yeVZpikXP6NJ8dBfzpC6oNU/JHu6johcBTyQpGsIWh8d7HSGDriHf9fDxECNqsaUfQDtdv34/3BsPDYfOkbSNbGYNk9TwZ37rxL5ooUgsqLeHYmPNsWNkLh7VlhFPfR7WhFSYzPKUOC2MWIvfpPVhMaWIsDHZi8yZ+3+/f2xg9r23x4MIxWdWWSgITAStqnlqVhgykE6YpsXwBBiaawXpSUmKNiYU8bUm5XKmZxbkBF0jSS6KEwHLiQs78+ejCIU4Lu49yOmUSQtN44d0g84NRiijwliMp0mxa2k9iEWKMP89AZDmdQBWNH2TNzGXHb09QtqT6khnqHbJ0bl36eiQW50zA9gENt/QOS1GXPuYUa6VvizT2xkFb/u8pdNKWlxLzIFFA8lzm3SjW0Dj4x3LaVHxLPSC8CzEWkDJXc5eiA3Rg6vwnKBdUL/SaJWzzmZz4piIlwdydxUtg8oV3z9znWQxYFoLDdRemlN0IK7Bas18zGrgd+6wKfmeftcRnogE6vFag+QwSgHWcR4q5WUWZ5+6HeCYY1QZ+1MJLxmSoFBhjdzSGrBR2EbPERkyiKyRIy2ZeRl2JCZxZTBrPOmhmi2XYVOB2jUL1riyEiSjdOkB5hSTOOU7nr1qgwHPvFohMlE3XrIhEOVcSDt7dYI2zuxF8VLKrNqEgjzbgRzFc2A3061aqgt9laQuXRWcQKhmufvpcqnB0aEVULeEic/R53VzxHqX7CE7vut9cQ19RuJM1PQbN7yqqsVX6U3AhIE3Hx6d+J4FJJthLWM7aQqA62c3H8zV/ZV3Nh9rBjdO0uyhYT5tqwRcdBOsqIpNrItutaErNJOzY1beT5lClFXht2h0MCzsQIl1+4JKCX2YW/yDbxunoMPvicPT9/cHDx6uf583kw+SjZD/Jki+Th8k3yZPkJCHJz8kvya/Jb73fe3/2/ur9vaRev7aacyfZenr//AfAZA5g</latexit>

basicsolution0

@1 0 0 �c3 �c4 �c5 b00 1 0 a13 a14 a15 b10 0 1 a23 a24 a25 b2

1

A

<latexit sha1_base64="mOj6wa+2ndz1QVomZekvPrk7apE=">AAAIpHicfVXrbts2FJa7rXO9W7r9GvpHqNsg29LAUlJsP9tmGDZgwbpL0gBhIlA0ZXGmSJWkGrsMH2EPuOfYA2yHsrzakh0CNo/4ffwOzzm8pCVn2oxGf/fuvPf+B3c/7N8bfPTxJ59+tnP/8zMtK0XoKZFcqvMUa8qZoKeGGU7PS0VxkXL6Kp0ee/zVG6o0k+IPMy/pZYEngmWMYANDyc5fqMDGkrrdEGcHUbgbjprfE5IcLrqjRfc03E2TEUIDjy6ZOLHRoWuMIzB8/9QPpEnUcJd8gOIlN264ccONBy7ZGY4ORnULu0bUGMOgaS+T+3f/RWNJqoIKQzjW+iIalebSYmUY4dQNUKVpickUT6jFhdbzInXhY4g5123MD27EZnWa3WDweGX0ojLZd5eWibIyVBCYCFhW8dDI0Cc6HDNFieFzMDBRDNYTkhwrTAyUY03KplJODU41uECCXhNZFFiMrS+Nu4gvLeI0M3sopRMmIDSF584OIzeMQ0SBtxgJkWKT3HzVFiFau4sIRBbTCWwW7YZRPXPx4dYnSJNTdc00dRYZOjM2fDfSFuesgF0KGnbhHZYir12bky2VfsrCtjcO2uJ/T/4j7HjJMfcSGSTPRs7GbQ2FvX8sJnXF19Q9wjch2gCSp3JmQ7SP9nWV/gnlgur5r3oJ63wmxq6uSE4wt+ftJTDxxtkr+yRqA4b54HC1CZPSrITlWZ3Z7xg13I19Wnq/02864tOiBjZ4LUHzChKAVTuPFHPdRJmm9rf2TDDKFfy4g+eMCV8pMBJ7nEBWMjNvs4qVmIpNIUFaVvMSb0qM50zbpGS6gabXWJpNCtytka/erYXQLcpmHaDcIIFTjsPZTQcs8MzZOSJjacIlq0WinEsBB++Rt5LoUQs+ztltm7Agz1fg5204MyvoD51Ueb+L0mY2ap1BqKR/YehrIf3RoSWRlYCLzNLXVf2SOBTuITi9y+/6Gvqewp2s6Alo/lJShY1UX4OLAtJ0cnLmthKYYAV7C8tZWghUx9v5eLbkN9btfKgd3Dj1/zYKVpO6WtCjfW/dRmRiSWTbFXWumIAd2/QbaRaVSoLX+n8Lw8AOhEgXHVxS8GRG7Qeya5zFB9HhQfzr0fDZi+bx7AcPgofBXhAF3wbPgh+Dl8FpQIJ/el/2HvaG/d3+z/3f+6cL6p1eM+eLYK31r/4Dd5Ei/w==</latexit>

13

STEP 2: CHOOSE ENTERING COLUMN

pick something with ci > 0

this guarantees objective will increase

0

BBBBBB@

b0b1b2000

1

CCCCCCA


basicsolution0

@1 0 0 �c3 �c4 �c5 b00 1 0 a13 a14 a15 b10 0 1 a23 a24 a25 b2

1

A


x4

14

STEP 3: CHOOSE LEAVING ROW

x4

0

@1 0 0 �c3 �c4 �c5 b00 1 0 a13 a14 a15 b10 0 1 a23 a24 a25 b2

1

A


0

BBBBBB@

b0b1b2000

1

CCCCCCA


basicsolution

pick row with minimal this guarantees all basic variables remain non-negative

bi/ari

15

STEP 4: ELIMINATE/PIVOT

this guarantees all basic variables remain non-negative

…rinse and repeat

0

@1 0 0 �c3 �c4 �c5 b00 1 0 a13 a14 a15 b10 0 1 a23 a24 a25 b2

1

A


basicsolution

0

@1 �c1 0 �c3 0 �c5 b00 a11 0 a13 1 a15 b10 a21 1 a23 0 a25 b2

1

A

<latexit sha1_base64="edDHUY0Vk+HCqwMg+cTMih9LEFE=">AAAIpHicfVXbcts2EKXSNlXVm9M+ZfrCiRKP2zgeUW6meUziTqedqafpxY5nDJsDQqCICgQYAIylwPiEfmC/ox/QLiiqlkjKeCB3cc7uYndxSQrOtBmN/u7dee/9D+5+2P9o8PEnn372+c69L061LBWhJ0Ryqc4SrClngp4YZjg9KxTFecLp62R25PHXb6nSTIo/zKKgFzmeCpYygg1MxTt/oRwbS6pxTZwdROFu+ITE/jdaioc34tNwN4lHCA28jmMbRa4GvXLoFTD08lMvJ3F0wx1HNVwph7Whl2vueODineHoYFSNsC1EtTAM6vEqvnf3XzSRpMypMIRjrc+jUWEuLFaGEU7dAJWaFpjM8JRanGu9yBMXPoKcM93E/GQnNq/K7AaDR2uz56VJn11YJorSUEHAELC05KGRoS90OGGKEsMXIGCiGKwnJBlWmBhox4Yrm0g5MzjREAIJekVknmMxsb417nx8YRGnqdlDCZ0yAakpvHB2GLnhOEQUeMuZECk2zczXTSdEa3cegZOlOYHNot0wqiyXits0kCaj6opp6iwydG5seDPTdM5ZDrsUfNhldFiKvHJNTrry9FMaNqNx8C3+j+SVsBUlw9y7SKF4NnJ23PShsI+PxbTq+IZ3j/AuRBtAskTObYj20b4ukz+hXdA9r1VL2OQzMXFVRzKCuT1rLoGJt85e2idREzDMJ4fLLkxKs5aWZ7WsbxgV3M59Vvi4s8ct57O8AjqiFuDzEgqAVbOOFHNdZ5kk9remJQjFGn7UwjPGhO8UCLE9iqEqqVk0WflaTnlXSlCW9bqMuwrjObMmKZ510PQGS7Npjts98t27tRG6Qen2A5RrJHDCcTi/boE5nju7QGQiTbhiNUiUcyng4D30Uhw9bMBHGbttE+bkxRr8ogmnZg39oVUqH3fZ2tRGjTMInfQvDH0jpD86tCCyFHCRWfqmrF4Sh8I9BKd3pVfX0PcU7mRFj8HnLwVV2Ej1DYTIoUzHx6duK4EJlrN3sJyVhMDrZDsfz1f8WrqdD72DG6f6bqNgNa26BX+076XbiEysiGy7R50pJmDH1v9OmkWFkhC1+m5hGNiBkOnyB5cUPJlR84FsC6fjg+jwYPzrt8PnL+vHsx98FTwI9oIo+C54HvwYvApOAhL807vfe9Ab9nf7P/d/758sqXd6tc2XwcboX/4HUmAi9g==</latexit>

0

BBBBBB@

b00b20b10

1

CCCCCCA

<latexit sha1_base64="ZILcUxHlaRgjs/WK7NbYKPVyCPs=">AAAIPXicfVXNjts2EFbSNE7dv0177EWIk2LbbhaWW6A9Jt0gaA6LpEV3s8ByI1A0ZbGmSIWksnYYvkGeJtcU6HP0AXIres2xHcp2YlPyCrA15PfxG84MOcoqzrQZDv++dPmDKx9e7V37qP/xJ59+9vnO9S+OtawVoUdEcqlOMqwpZ4IeGWY4PakUxWXG6eNseuDxx8+o0kyK3828omclngiWM4INTKU7X6MSG0uc7WfpEKG+/2Xp6J2VeKvv0p3BcH/YPHHbSJbGIFo+j9LrV/9DY0nqkgpDONb6NBlW5sxiZRjh1PVRrWmFyRRPqMWl1vMyc/Et2EyhQ8xPdmKzJn7X799amz2tTf7TmWWiqg0VBBYCltc8NjL2GYjHTFFi+BwMTBSD/cSkwAoTA3nakLKZlFODMw0ukKDnRJYlFmPrc+ZOR2cWcZqbXZTRCRMQmsJzZweJG4xiRIG3mImRYpPCfBOKEK3daQIii+UEqqjdIGlWLgZuc4E0BVXnTFNnkaEzY+P3M6E4ZyUcH9CwC++wFXnuQk6+UnqQx6E3DtrinSc/iFteCsy9RA7Js4mzo1BDYe8fi0lT8Q11j/AuRBtAikzObIz20J6usz+gXFA9P2q2sMlnYuyaihQEc3sSboGJZ84+sbeTEDDMB4frLkxKsxaWZ7VWv2c0cDv2aeX9Tr9riU/LBujwWoHmE0gAVmEeKeZ6GWWW2d/ClWBUa/hBCy8YE75SYKT2IIWs5GYessq1mMqukCAt63kZdSXGc6YhKZ120PQGS7NJids18tW7sBA6oHTrAOUFEjjjOJ69aIElnjk7R2QsTbxiBSTKuRRw8W56K01uBvBBwS46hCW5uwbfDeHcrKH3W6nyfhelzW0S3EGopG/99KmQ/urQishaQCOz9GndtHiH4l0Et3c1btrQPQo9WdFD0HxYUYWNVN+CixLSdHh47LYSmGAlew7bWVkIVMfb+Xi24i+ti/lQO+g4zf82ClaTplrwRnveuojIxIrItivqQjEBJ3b57qRZVCkJXpv/LQwDJxAiXbygScEnMwk/kG3jeLSffL8/+vWHwZ2flx/Pa9FX0Y1oN0qiH6M70S/Ro+goItHL6FX0Ovqz91fvTe+f3r8L6uVLyzVfRhtP7+3/vPoOYA==</latexit>

pick row with minimal bi/ari

16

PHASE 1 PROBLEM

maximize cTx

subject to Ax = b

x � 0

want to solve this

Need feasible solution: solve “phase 1” problem

minimize t

subject to Ax = b

x � �t

t � 0

measureinfeasibility

17

TWO APPROACHESTwo phase approach

maximize cTx

subject to Ax = b

x � 0

minimize t

subject to Ax = b

x � �t

t � 0

then

“Big M” approach

phase 1 phase 1I

maximize cTx�Mt

subject to Ax = b

x � �t

t � 018

REINFORCEMENT LEARNING

Space Invaders Montezuma’s revenge

19

STATE SPACE

You start in a STATE

20

STATE SPACE


You choose an ACTION

21

STATE SPACE



$

Get a REWARD

22

STATE SPACE



$

Get a REWARD

VALUE of a state

Vs =1X

k=0

�krsk,ak

<latexit sha1_base64="+bWunOnYp/ym02ceoSryUY+aQVY=">AAACLnicbVDLSsNAFJ34tr6qLt0MFsWFlKQKuimIblwq2Ac0bbiZTuqQmUmYmQgl9C/8ENdu9RsEF+JW/8JJ7cKqBy4czrmXe+8JU860cd1XZ2Z2bn5hcWm5tLK6tr5R3txq6iRThDZIwhPVDkFTziRtGGY4baeKggg5bYXxReG37qjSLJE3ZpjSroCBZBEjYKwUlKvNQOM69nUmgjyuu6Oez2RkhtgfgBDQi/E+VkGug/gQgngUlCtu1R0D/yXehFTQBFdB+dPvJyQTVBrCQeuO56amm4MyjHA6KvmZpimQGAa0Y6kEQXU3H/81wntW6eMoUbakwWP150QOQuuhCG2nAHOrf3uF+J/XyUx02s2ZTDNDJfleFGUcmwQXIeE+U5QYPrQEiGL2VkxuQQExNspSaWqPZIRG1iqy8X4n8Zc0a1XvqFq7Pq6cnU9SWkI7aBcdIA+doDN0ia5QAxF0jx7RE3p2HpwX5815/26dcSYz22gKzscXBiOoNQ==</latexit> 23

STATE SPACE$

Bellman equation:value of optimal policy

VALUE of a state

Vs = maxa

{rs,a + � ⇤ Vs,a}<latexit sha1_base64="KEqmwhE+zpSLyCMkA6+j43przdY=">AAACLHicbVDLSsNAFJ34Nr6qLt0MFkG0lEQF3QiiG5cKthWaEm6mk3ZwZhJmJmIJ+Qk/xLVb/QY3Im4F/8Jpm4WvAxcO59zLvfdEKWfaeN6rMzE5NT0zOzfvLiwuLa9UVteaOskUoQ2S8ERdR6ApZ5I2DDOcXqeKgog4bUU3Z0O/dUuVZom8MoOUdgT0JIsZAWOlsFJrhhof40DAXQg4yFWY6xoUeDfogRCAd3CzVILCDStVr+6NgP8SvyRVVOIirHwG3YRkgkpDOGjd9r3UdHJQhhFOCzfINE2B3ECPti2VIKju5KOvCrxllS6OE2VLGjxSv0/kILQeiMh2CjB9/dsbiv957czER52cyTQzVJLxojjj2CR4GBHuMkWJ4QNLgChmb8WkDwqIsUG67o89khEaW6uw2fi/k/hLmnt1f7++d3lQPTktU5pDG2gTbSMfHaITdI4uUAMRdI8e0RN6dh6cF+fNeR+3TjjlzDr6AefjC3tupjg=</latexit>

Vs =1X

k=0

�krsk,ak

<latexit sha1_base64="+bWunOnYp/ym02ceoSryUY+aQVY=">AAACLnicbVDLSsNAFJ34tr6qLt0MFsWFlKQKuimIblwq2Ac0bbiZTuqQmUmYmQgl9C/8ENdu9RsEF+JW/8JJ7cKqBy4czrmXe+8JU860cd1XZ2Z2bn5hcWm5tLK6tr5R3txq6iRThDZIwhPVDkFTziRtGGY4baeKggg5bYXxReG37qjSLJE3ZpjSroCBZBEjYKwUlKvNQOM69nUmgjyuu6Oez2RkhtgfgBDQi/E+VkGug/gQgngUlCtu1R0D/yXehFTQBFdB+dPvJyQTVBrCQeuO56amm4MyjHA6KvmZpimQGAa0Y6kEQXU3H/81wntW6eMoUbakwWP150QOQuuhCG2nAHOrf3uF+J/XyUx02s2ZTDNDJfleFGUcmwQXIeE+U5QYPrQEiGL2VkxuQQExNspSaWqPZIRG1iqy8X4n8Zc0a1XvqFq7Pq6cnU9SWkI7aBcdIA+doDN0ia5QAxF0jx7RE3p2HpwX5815/26dcSYz22gKzscXBiOoNQ==</latexit>

24

STATE SPACE$

LP formulation

subject to<latexit sha1_base64="2a460Gc7i3re4XqlAIunpE6kndM=">AAACCHicbVC7TsNAEFyHVzCvACWNRYREFdmhgDKChjJI5CESKzpf1smR89m6OyNFVn6Amha+gQ7R8hd8An/BJXFBEkZaaTSzq92dIOFMadf9tgpr6xubW8Vte2d3b/+gdHjUVHEqKTZozGPZDohCzgQ2NNMc24lEEgUcW8HoZuq3nlAqFot7PU7Qj8hAsJBRoo30oNLgEal2dNwrld2KO4OzSryclCFHvVf66fZjmkYoNOVEqY7nJtrPiNSMcpzY3VRhQuiIDLBjqCARKj+bXTxxzozSd8JYmhLamal/JzISKTWOAtMZET1Uy95U/M/rpDq88jMmklSjoPNFYcrNg870fafPpPmXjw0hVDJzq0OHRBKqTUi2vbBHMIqhsSYmG285iVXSrFa8i0r1rlquXecpFeEETuEcPLiEGtxCHRpAQcALvMKb9Wy9Wx/W57y1YOUzx7AA6+sX3YOaGQ==</latexit>

minv

1nT v

<latexit sha1_base64="nIIE6cAQJRLwmNm+Nw4aEKX2eW8=">AAACH3icbVC7TsMwFHV4lvIKMDJgUSExVUlBgrGChbFIfUlNiRzHaa3aTrCdSlWUkQ9hZoVvYEOs/QT+Avcx0JYjXenonHt17z1BwqjSjjO21tY3Nre2CzvF3b39g0P76Lip4lRi0sAxi2U7QIowKkhDU81IO5EE8YCRVjC4n/itIZGKxqKuRwnpctQTNKIYaSP59pnHqfCH0HtOUQihx5HuB1Hm+iJ/qsMh9O2SU3amgKvEnZMSmKPm2z9eGOOUE6ExQ0p1XCfR3QxJTTEjedFLFUkQHqAe6RgqECeqm00fyeGFUUIYxdKU0HCq/p3IEFdqxAPTOTlULXsT8T+vk+rotptRkaSaCDxbFKUM6hhOUoEhlQRrNjIEYUnNrRD3kURYm+yKxYU9gmISGSs32bjLSaySZqXsXpUrj9el6t08pQI4BefgErjgBlTBA6iBBsDgBbyBd/BhvVqf1pf1PWtds+YzJ2AB1vgXXduiMw==</latexit>

Inequality constraintfor every (s,a,r) observation


Vs = maxa


vs � rs,a + �Vs,a, 8(s, a, r)<latexit sha1_base64="zJDWTmtv2bGcw8p+XLa+zZmdsvE=">AAACMXicbVDLSgMxFM34tr6qLt0Ei6BYykwVdFl041LBtkJbhjvpnRpMMkOSKZSh3+GHuHar3+BO3Ik/YfpYqPVA4Nxz7uXenCgV3Fjff/Pm5hcWl5ZXVgtr6xubW8XtnYZJMs2wzhKR6LsIDAqusG65FXiXagQZCWxGD5cjv9lHbXiibu0gxY6EnuIxZ2CdFBaDfmhou4dUh7kpw5AeuwqkBNqYCGXajhMNQtBDV5b1UVgs+RV/DDpLgikpkSmuw+Jnu5uwTKKyTIAxrcBPbScHbTkTOCy0M4MpsAfoYctRBRJNJx9/bUgPnNKl7gL3lKVj9edEDtKYgYxcpwR7b/56I/E/r5XZ+LyTc5VmFhWbLIozQW1CRznRLtfIrBg4Akxzdytl96CBWZdmofBrj+IMY2cNXTbB3yRmSaNaCU4q1ZvTUu1imtIK2SP75JAE5IzUyBW5JnXCyCN5Ji/k1Xvy3rx372PSOudNZ3bJL3hf31LYqCY=</latexit>

25

STATE SPACE

LP formulation

subject to<latexit sha1_base64="2a460Gc7i3re4XqlAIunpE6kndM=">AAACCHicbVC7TsNAEFyHVzCvACWNRYREFdmhgDKChjJI5CESKzpf1smR89m6OyNFVn6Amha+gQ7R8hd8An/BJXFBEkZaaTSzq92dIOFMadf9tgpr6xubW8Vte2d3b/+gdHjUVHEqKTZozGPZDohCzgQ2NNMc24lEEgUcW8HoZuq3nlAqFot7PU7Qj8hAsJBRoo30oNLgEal2dNwrld2KO4OzSryclCFHvVf66fZjmkYoNOVEqY7nJtrPiNSMcpzY3VRhQuiIDLBjqCARKj+bXTxxzozSd8JYmhLamal/JzISKTWOAtMZET1Uy95U/M/rpDq88jMmklSjoPNFYcrNg870fafPpPmXjw0hVDJzq0OHRBKqTUi2vbBHMIqhsSYmG285iVXSrFa8i0r1rlquXecpFeEETuEcPLiEGtxCHRpAQcALvMKb9Wy9Wx/W57y1YOUzx7AA6+sX3YOaGQ==</latexit>

minv

1nT v

<latexit sha1_base64="nIIE6cAQJRLwmNm+Nw4aEKX2eW8=">AAACH3icbVC7TsMwFHV4lvIKMDJgUSExVUlBgrGChbFIfUlNiRzHaa3aTrCdSlWUkQ9hZoVvYEOs/QT+Avcx0JYjXenonHt17z1BwqjSjjO21tY3Nre2CzvF3b39g0P76Lip4lRi0sAxi2U7QIowKkhDU81IO5EE8YCRVjC4n/itIZGKxqKuRwnpctQTNKIYaSP59pnHqfCH0HtOUQihx5HuB1Hm+iJ/qsMh9O2SU3amgKvEnZMSmKPm2z9eGOOUE6ExQ0p1XCfR3QxJTTEjedFLFUkQHqAe6RgqECeqm00fyeGFUUIYxdKU0HCq/p3IEFdqxAPTOTlULXsT8T+vk+rotptRkaSaCDxbFKUM6hhOUoEhlQRrNjIEYUnNrRD3kURYm+yKxYU9gmISGSs32bjLSaySZqXsXpUrj9el6t08pQI4BefgErjgBlTBA6iBBsDgBbyBd/BhvVqf1pf1PWtds+YzJ2AB1vgXXduiMw==</latexit>

Inequality constraintfor every (s,a,r) observation


Vs = maxa


vs � rs,a + �Vs,a, 8(s, a, r)<latexit sha1_base64="zJDWTmtv2bGcw8p+XLa+zZmdsvE=">AAACMXicbVDLSgMxFM34tr6qLt0Ei6BYykwVdFl041LBtkJbhjvpnRpMMkOSKZSh3+GHuHar3+BO3Ik/YfpYqPVA4Nxz7uXenCgV3Fjff/Pm5hcWl5ZXVgtr6xubW8XtnYZJMs2wzhKR6LsIDAqusG65FXiXagQZCWxGD5cjv9lHbXiibu0gxY6EnuIxZ2CdFBaDfmhou4dUh7kpw5AeuwqkBNqYCGXajhMNQtBDV5b1UVgs+RV/DDpLgikpkSmuw+Jnu5uwTKKyTIAxrcBPbScHbTkTOCy0M4MpsAfoYctRBRJNJx9/bUgPnNKl7gL3lKVj9edEDtKYgYxcpwR7b/56I/E/r5XZ+LyTc5VmFhWbLIozQW1CRznRLtfIrBg4Akxzdytl96CBWZdmofBrj+IMY2cNXTbB3yRmSaNaCU4q1ZvTUu1imtIK2SP75JAE5IzUyBW5JnXCyCN5Ji/k1Xvy3rx372PSOudNZ3bJL3hf31LYqCY=</latexit>

Add an equation every timeyou explore the environment

26

MODIFYING THE PROBLEM

x0

Ax b

c

27


x0

Ax b

c

new constraint

Note: this happens all the time in reinforcement learning28


x0

Ax b

c

new constraintphase I solution

is far away

can’t we just “snap” back onto the simplex?29

DUAL SIMPLEX METHODprimal problem

Lagrangian

minimize cTx

subject to Ax = b

x � 0

max�,⌘0

minx

cTx+ h�, Ax� bi+ h⌘, xi

dual

max�,⌘0

minx

hc+AT�+ ⌘, xi � h�, bior

= 0

maximize � bT�

subject to AT� � �c<latexit sha1_base64="w4DG4gs608BAysoKJBGL/0lo3fk=">AAAIS3icfVXdbiM1FJ5d2E0JP9uFS24ssosKtFUmrASXW4oQXFQsaNutVLeRx/FkTDz2rO3ZJnj9IjwNtyDxADwHdwgJjicJTTxJ52Lm2N93vjPnHP9kleDG9vt/3rn7xpv37nd23uq+/c677z3Yffj+mVG1puyUKqH0eUYME1yyU8utYOeVZqTMBHuRTY4D/uIV04Yr+dzOKnZZkrHkOafEwtRw9wkuyZSX/GeGPkYH2dVzhAV4jwjGXWwsTB7dzCE8ZuiAouFur3/Ybx7UNtKF0UsWz7Phw/v/4pGidcmkpYIYc5H2K3vpiLacCua7uDasInRCxsyR0phZmXn0uCS2MDEWJjdi06Yavtt9vDJ7Udv8y0vHZVVbJik4ApbXAlmFQj3QiGtGrZiBQajm8D+IFkQTaqFqa1IuU2piSWYgBJbsmqqyJHLkoILWXwwuHRYst3s4Y2MuITVNZt71Ut8bIMyAN59BWPNxYT+JRagx/iIFkbk7hZ4a30sbz/nArzsoWzB9zQ3zDls2tQ7dzMTigpewmEDDzaPDr6hrH3PypdJ3OYqjCdCW/0cKA9SKUhARJHIonku9G8QamoT4RI6bjq+pB0RsQowFpMjU1CG8j/dNnf0E7YLuhVHzC+t8Lke+6UhBiXDn8S9w+cq7K3eQxoDlITlSb8KUsitpBVbL+4bRwO3cJ1WIO/msJT4pG2BD1Ao0r6AARMd1ZESYRZZZ5n6MPcGoVvDjFl5wLkOnwBi64yFUJbezmFWu5FRuSgnKslqXwabCBM4kJg0nG2hmjWX4uCTtHoXu3doIE1E26wDlNZYkEwRNX7dAOBC9m2E6UhYtWRGJCaEkbLxHwRqmjyL4uOC3LcKSHq3ARzGc2xX0m1apQtx5a3OXRnsQOhkuAvZSqrB1WEVVLeEgc+xl3Rz4HqM9DLt3OW6Ooa8ZnMmanYDm9xXTxCr9KYQooUwnJ2d+K4HL5uLwbmlhUB1t5y8uGuAvrNv50Ds4cZr3NgrR46Zb8MX7wbqNyOWSyLcrmkJzCSt28d1Ic7jSCqI27y0MCysQMp1/4JCCKzONL8i2cTY4TD8/HPzwpPf0q8XluZN8mHyU7CVp8kXyNPk2eZacJjT5Jfk1+S35vfNH56/O351/5tS7dxY+HyRrz869/wBhVxNP</latexit>

30

DUAL SIMPLEX METHODprimal problemminimize cTx

subject to Ax = b

x � 0

dual

Adding a constraint (row) to the primal is like adding a variable (column) to the dual!

maximize � bT�

subject to AT� � �c<latexit sha1_base64="w4DG4gs608BAysoKJBGL/0lo3fk=">AAAIS3icfVXdbiM1FJ5d2E0JP9uFS24ssosKtFUmrASXW4oQXFQsaNutVLeRx/FkTDz2rO3ZJnj9IjwNtyDxADwHdwgJjicJTTxJ52Lm2N93vjPnHP9kleDG9vt/3rn7xpv37nd23uq+/c677z3Yffj+mVG1puyUKqH0eUYME1yyU8utYOeVZqTMBHuRTY4D/uIV04Yr+dzOKnZZkrHkOafEwtRw9wkuyZSX/GeGPkYH2dVzhAV4jwjGXWwsTB7dzCE8ZuiAouFur3/Ybx7UNtKF0UsWz7Phw/v/4pGidcmkpYIYc5H2K3vpiLacCua7uDasInRCxsyR0phZmXn0uCS2MDEWJjdi06Yavtt9vDJ7Udv8y0vHZVVbJik4ApbXAlmFQj3QiGtGrZiBQajm8D+IFkQTaqFqa1IuU2piSWYgBJbsmqqyJHLkoILWXwwuHRYst3s4Y2MuITVNZt71Ut8bIMyAN59BWPNxYT+JRagx/iIFkbk7hZ4a30sbz/nArzsoWzB9zQ3zDls2tQ7dzMTigpewmEDDzaPDr6hrH3PypdJ3OYqjCdCW/0cKA9SKUhARJHIonku9G8QamoT4RI6bjq+pB0RsQowFpMjU1CG8j/dNnf0E7YLuhVHzC+t8Lke+6UhBiXDn8S9w+cq7K3eQxoDlITlSb8KUsitpBVbL+4bRwO3cJ1WIO/msJT4pG2BD1Ao0r6AARMd1ZESYRZZZ5n6MPcGoVvDjFl5wLkOnwBi64yFUJbezmFWu5FRuSgnKslqXwabCBM4kJg0nG2hmjWX4uCTtHoXu3doIE1E26wDlNZYkEwRNX7dAOBC9m2E6UhYtWRGJCaEkbLxHwRqmjyL4uOC3LcKSHq3ARzGc2xX0m1apQtx5a3OXRnsQOhkuAvZSqrB1WEVVLeEgc+xl3Rz4HqM9DLt3OW6Ooa8ZnMmanYDm9xXTxCr9KYQooUwnJ2d+K4HL5uLwbmlhUB1t5y8uGuAvrNv50Ds4cZr3NgrR46Zb8MX7wbqNyOWSyLcrmkJzCSt28d1Ic7jSCqI27y0MCysQMp1/4JCCKzONL8i2cTY4TD8/HPzwpPf0q8XluZN8mHyU7CVp8kXyNPk2eZacJjT5Jfk1+S35vfNH56/O351/5tS7dxY+HyRrz869/wBhVxNP</latexit>

31

DUAL SIMPLEX METHOD0

@1 �c1 0 �c3 0 �c5 00 a11 0 a13 1 a15 b10 a21 1 a23 0 a25 b2

1

A

solve dual simplex tableau

put new row here0

@1 �c1 0 �c3 0 �c5 �c6 00 a11 0 a13 1 a15 a16 b10 a21 1 a23 0 a25 a16 b2

1

A

basic/feasible solution is still feasible32

COMPLEXITY

Klee and Minty: “How good in the Simplex algorithm?” 1972

Simplex has exponential worst-case behavior

Is there a polynomial time algorithm for LP?

Holds for most common pivot rules

Klee-Minty Cube

Average case complexityis polynomial for random

constraints

33

L15 linear programming - University Of Marylandtomg/course/cmsc764/L15... · 1948: Soviets blockade...

Documents

Transcript of L15 linear programming - University Of Marylandtomg/course/cmsc764/L15... · 1948: Soviets blockade...