1 DYNAMICPROGAMMINGOPTIMIZINGOVERTIME

FINDTHESHORTESTPATH FROMTHE ROOTLOWESTcost

r r n r

I s l 0

7 T

E S 4 Lauer

Geer g 7 Craw

SOLVEDBACKTO FRONT GENERALSOLUTIONTO

SHORTESTPATHS IsPROBLEMREPEATS ITSELF ie SOLVE BELLMANFORD J

L MIN Cat La BELLMAN CQIae LEFTRIGHT

ABSTRACT DEFINITION

Tti

i

Xo

I _floe aPOLICY ITCH OR ItOBJECTIVET MAXIMIZE sum of REWARDS

type MAX rGet it r XT OVER Ito Ita c A

pVALUEFUNCTION 126,1T

T

TIEBELLMNEQUATOITHEOREM Vtc aMeAax rcx.asi V.fi VCx rk

PROOF

Veluach q

g.arC4aorG4iaDt trGee.iiati trCxta

an iMAX rbc da t MAX r x 9 t t rGen.ae trGeta CA 9 __ atyEA

Vt Cic

ma.A.yafrhia.ltVetch D

SOME CODE

THE PRINCIPLEOFOPTIMALITYaka WHENSHOULDDYNAMICPROGRAMMINGWork

EXAMPLEGHORTEST LONGESTPATHS PATHGoes

m are.EEuEIFaraj

77 D 7 7u

JrJr 77L 7C 7

THE CURSE OF DIMENSIONALITY

TRIEHAIRE AGAIN

T112 I

i STATESTooBIG

RUBIK'sCUBE

43 1019

A SHORTESTPATHPROBLEMBUTOPTIMALSolutionUNKNOWN

JieweknowHowtoFINDTHEoptimesource SolutionButComputersAren'tFastcwoundwww.oeoia

R

T

OTHEROBSERVATIONS

MINIMIZE MIN CGE.ae 404 over a at eA

BELLMAN CQQLecoq Maley CGctia 1 Luck

MEMOIZE SOMETIMES PROBLEM REPEATS ez I 0 20 20 1013 of 13so STORE VALUES ACTIONS 10 Oz

TELPSOVERCOME CURSEOFDIMENSIONALITY 10 3

GENERALIZE DP STILLWORKSFOR

MAX II EGG at Ceti Flat

5 t AoCHo AT1EATI

EXAMPLE AN INVESTOR HAS A FUNDIT HAS POUNDS ATTIMEZERO

MONEY CAN'T BE WITHDRAWNITPAYS INTEREST r x 100 PERYEARFORT YEARSTHE INVESTOR CONSUMES PROPORTION at OF INTEREST REINVESTSTHERES

WHAT SHOULD YOUDOTO MAXIMIZE CONSUMPTIONOVERTYEARS

ANGIER LET It TIME t FOND VALUE

WANT TO SOLVE BACKWARDS INTIMET l

x MAX E rxt.atT t o

S t 04 1 04 t r xt tatINTERESTNOTCONSUMED

OVER 0 E At E I

IF IT'STHE LASTYEAR CONSUME EVERYTHING

At 1 1 V se r x

IF 2 YEARS LEFT

VGd 1k roca Y aDrx tix ta

roc MAX aG r t Hr roc 2 Hr03 a El

p f e

or

CALLED BANGBANGCONTROL

LETS SOLVEFOR t.tlYEARS LEFTA GUESS

ex roc ft FOR SOME

ftPROOF BY INDUCTION

VtoW roca Vt xtructa

p x trace a I

rxffta.ve all re pathr x peer IvHp

70 ATS

fu rx f Hr V Hp

APICTURE plur Hen f I pCHr

x Pcooisonex

x

xx

x I a e

re'tii.e.li

eq1q 2fs s fsti Gtrfs fe psCt.yt.s

OPTIMAL TO CONSUMEFOR LAST Lrt YEARS

OTHERWISE REINVEST ALL INTEREST

SUMMARY

DP Vecoco MAX rheoao 1 rlxt i.at tr xtao at cA

BELLMAN EQI yea rarityr x a tf I V x RG

BELLMAN CQQ CAN BESOLVED INDUCTIVELY t o 1,2

BUTLOTS OFTRICKSEXISTCOMPLEXITY SCALESBADLYLeg

memoizing function approximation

PRINCIPLEOF OPTIMALITY ToSOLVEANOPTIMIZATION

ALWAYSTAKEOPTIMALDESCIONS FROMEACHSTATEONWARDS