Discrete Dynamic Optimization: Six Examples

Dr. Tai-kuang Ho�

�Associate Professor. Department of Quantitative Finance, National Tsing Hua University, No.101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan 30013, Tel: +886-3-571-5131, ext. 62136, Fax:+886-3-562-1823, E-mail: tkho@mx.nthu.edu.tw.

Example 1: the neoclassical growth model

� Uhlig (1999), section 4

� Model principles

� Specify the environment explicitly:

1. Preferences

2. Technologies

3. Endowments

4. Information

� State the object of study:

1. The social planner�s problem

2. The competition equilibrium

3. The game

� The environment:

� Preferences: the representative agent experiences utility according to

U = Et

24 1Xt=0

�tC1��t � 11� �

35

� 0 < � < 1

� � > 0

� Absolute risk aversion: �u00(c)u0(c)

� Relative risk aversion: �cu00(c)

u0(c)

� � is the coe¢ cient of relative risk aversion

� Technologies: we assume a Cobb-Douglas production function

Yt = ZtK�tN

1��t

� Budget constraint:

Kt+1 = It + (1� �)Kt

Ct + It = Yt

� or equivalently

Ct +Kt+1 = ZtK�tN

1��t + (1� �)Kt

� 0 < � < 1

� 0 < � < 1

� Zt, the total factor productivity, is exogenously evolving according to

logZt = (1� ) log �Z + logZt�1 + �t

�t � i:i:d:N�0;�2

�

� 0 < < 1

� Endowment:

� Nt = 1

� K0

� Information: Ct, Nt, and Kt+1 need to be chosen based on all information Itup to time t.

� The social planner�s problem

� The objective of the social planner is to maximize the utility of the representativeagent subject to feasibility,

max(Ct;Kt)

1t=0

Et

24 1Xt=0

�tC1��t � 11� �

35

s:t: K0; Z0

Ct +Kt+1 = ZtK�t + (1� �)Kt

logZt = (1� ) log �Z + logZt�1 + �t

�t � i:i:d:N�0;�2

�

� To solve it, form the Lagrangian:

L = max(Ct;Kt+1)

1t=0

Et

1Xt=0

24�tC1��t � 11� �

+ �t�t�ZtK

�t + (1� �)Kt � Ct �Kt+1

�35

� The �rst order conditions are:

@L

@�t: 0 = Ct +Kt+1 � ZtK

�t � (1� �)Kt

@L

@Ct: 0 = C

��t � �t

@L

@Kt+1: 0 = ��t + �Et

h�t+1

��Zt+1K

��1t+1 + (1� �)

�i

� Notice that Kt+1 appears in period t and period t+ 1.

� The �rst order conditions are often also called Euler equations.

� One also obtains the transversality condition.

0 = limT!1

E0h�TC

��T KT+1

i

� The transversality condition is a limiting Kuhn-Tucker condition.

� It means that in net present value terms, the agent should not have capital leftover at in�nity.

� Rewrite the necessary conditions:

Ct = ZtK�t + (1� �)Kt �Kt+1 (1)

Rt = �ZtK��1t + (1� �) (2)

1 = Et

"�

Ct

Ct+1

!�Rt+1

#(3)

logZt = (1� ) log �Z + logZt�1 + �t; �t � i:i:d:N�0;�2

�(4)

� Equation accounting.

� To solve for the steady state, dropping the time indices and yield

�C = �Z �K� + (1� �) �K � �K

�R = � �Z �K��1 + (1� �)

1 = � �R

� From the welfare theorems, the solution to the competitive equilibrium yields thesame allocation as the solution to the social planner�s problem.

� The case for competitive equilibrium is provided in Uhlig (1999) section 4.4.

Example 2: Hansen�s real business cycle model

� Uhlig (1999), section 4

� Hansen�s real business cycle model

� The model is an extension of the stochastic neoclassical growth model.

� The main di¤erence is to endogenize the labour supply.

� The social planner solves the problem of the representative agent:

maxEt

1Xt=1

�t

0@C1��t � 11� �

�ANt

1A

s:t:

Ct + It = Yt

Kt+1 = It + (1� �)Kt

Yt = ZtK�tN

1��t

logZt = (1� ) log �Z + logZt�1 + �t; �t � i:i:d:N�0;�2

�

� Hansen only consider the case � = 1, so that the objective function is:

Et

1Xt=1

�t (logCt �ANt)

� Proof:

lim��!1

C1��t � 11� �

= lim��!1

e(1��) lnCt � 11� �

= lim��!1

@he(1��) lnCt � 1

i=@�

@ [1� �] =@�

= lim��!1

e(1��) lnCt � (� lnCt)�1

=e0 � (� lnCt)

�1= lnCt

� To solve it, form the Lagrangian:

L = max(Ct;Nt;Kt+1)

1t=0

Et

1Xt=0

2664 �t C1��t �11�� �ANt

!+�t�t

�ZtK

�tN

1��t + (1� �)Kt � Ct �Kt+1

�3775

� The �rst order conditions are:

@L

@Ct= C

��t � �t = 0

@L

@Nt= �A+ �t (1� �)ZtK

�tN

��t = 0

@L

@Kt+1= ��t + �Et�t+1

��Zt+1K

��1t+1N

1��t+1 + (1� �)

�= 0

� After arrangement

A = C��t (1� �)

Yt

Nt(1)

Rt = �Yt

Kt+ 1� � (2)

1 = �Et

" Ct

Ct+1

!�Rt+1

#(3)

� The steady state for the real business cycle model above is obtained by droppingthe time subscripts and stochastic shocks in the equations above.

A = �C�� (1� �)�Y�N

1 = � �R

�R = ��Y�K+ 1� �

� Exercise: Equation accounting.

Example 3: Brock-Mirman growth model I

� Chow (1997), chapter 2

� Consider the Brock-Mirman growth model:

maxfctg

Et

1Xt=0

�t ln ct

s:t:

kt+1 = k�t zt � ct

� The Lagrangian is:

L = Et

1Xt=0

h�t ln ct + �t�t (k

�t zt � ct � kt+1)

i

� The �rst order conditions are:

@L

@ct=

�t1

ct� �t�t

!= 0

@L

@kt+1= ��t�t + �t+1Et

��t+1�k

��1t+1 zt+1

�= 0

� The �rst order conditions can be rearranged as:

1

ct= �t

�t = ��Et��t+1k

��1t+1 zt+1

�

� Solve the problem explicitly.

� kT+1 = 0

� cT = k�TzT

� ct = d � k�t zt (guess a solution)

Et��t+1k

��1t+1 zt+1

�= Et

1

ct+1k��1t+1 zt+1

!

= Et

1

d � k�t+1zt+1k��1t+1 zt+1

!

= Et

1

d

1

kt+1

!=1

d

1

k�t zt � ct

�t = ��Et��t+1k

��1t+1 zt+1

�, 1

ct= ��

1

d

1

k�t zt � ct

ct =d

��(k�t zt � ct)

d � k�t zt =d

��(k�t zt � d � k�t zt)

1 =1

��(1� d)

d = 1� ��

) ct = d � k�t zt = (1� ��) k�t zt

Example 4: Brock-Mirman growth model II

� Chow (1997), chapter 2

� We shall use dynamic programming to solve the Brock-Mirman growth model.

� Consider the Brock-Mirman growth model:

maxfctg

Et

1Xt=0

�t ln ct

s:t:

kt+1 = k�t zt � ct

� Control variable: ct

� State variables: kt; zt

� The Bellman equation is:

V (kt; zt) = maxct[ln ct + �EtV (kt+1; zt+1)]

� We conjecture value function takes the form:

V (kt; zt) = a+ b ln kt + c ln zt

� a, b, c are coe¢ cients to be determined.

� EtV (kt+1; zt+1)

V (kt+1; zt+1) = a+ b ln kt+1 + c ln zt+1

= a+ b ln (k�t zt � ct) + c ln zt+1

* Et ln zt+1 = 0

) EtV (kt+1; zt+1) = a+ b ln (k�t zt � ct)

� It follows

V (kt; zt) = maxct[ln ct + �EtV (kt+1; zt+1)]

= maxctfln ct + � [a+ b ln (k�t zt � ct)]g

� maxctfg

� First order condition.

@ fg@ct

=1

ct� �b

k�t zt � ct= 0

) ct =k�t zt

1 + �b

� Substitute this optimal ct =k�t zt1+�b into maxct

fg and equate the expression to thevalue function, we can solve the coe¢ cients a, b, c, which after some algebraicmanipulation are:

a = (1� �)�1hln (1� ��) + �� (1� ��)�1 ln (��)

i

b = � (1� ��)�1

c = (1� ��)�1

Example 5: the money-in-the-utility function (MIU)

model

� Walsh (2003), chapter 2

� Utility function, labour supply, and production function.

u�ct;mt;lt

�=C1��t

1� �+

l1��t

1� �

Ct �hac1�bt + (1� a)m1�bt

i 11�b

nt = 1� lt

yt = eztk�t n1��t = f (kt; nt; zt)

zt = �zt�1 + et

� Two state variables.

at � � t +(1 + it�1) bt�1

1 + �t+mt�11 + �t

kt

� The objective function is:

Et

1Xt=0

�tu�ct;mt;1� nt

�

s:t:

f (kt; nt; zt) + (1� �) kt + at = ct + kt+1 +mt + bt

� The Bellman equation is:

V (at; kt) = maxhu�ct;mt;1� nt

�+ �EtV (at+1; kt+1)

i

� Use the de�nition of at to substitute for at+1.

at+1 � � t+1 +(1 + it) bt

1 + �t+1+

mt

1 + �t+1

� Use the budget constraint to eliminate kt+1.

kt+1 = f (kt; nt; zt) + (1� �) kt + at � ct �mt � bt

� Rewrite the Bellman equation as:

V (at; kt) = maxhu�ct;mt;1� nt

�+ �EtV (at+1; kt+1)

i

= max

26664u�ct;mt;1� nt

�+�EtV

0@ � t+1 +(1+it)bt1+�t+1

+ mt1+�t+1

; f (kt; nt; zt)

+ (1� �) kt + at � ct �mt � bt

1A37775

= maxct;bt;mt;nt

fg

� The �rst order conditions are:

@ fg@ct

: uc�ct;mt;1� nt

�� �EtVk (at+1; kt+1) = 0

@ fg@bt

: �Et

1 + it

1 + �t+1

!Va (at+1; kt+1)� �EtVk (at+1; kt+1) = 0

@ fg@mt

: um�ct;mt;1� nt

�+�Et

1

1 + �t+1

!Va (at+1; kt+1)��EtVk (at+1; kt+1) = 0

@ fg@nt

: �un�ct;mt;1� nt

�+ �EtVk (at+1; kt+1) fn (kt; nt; zt) = 0

@V (at; kt)

@at=@ fg@at

: Va (at; kt) = �EtVk (at+1; kt+1)

@V (at; kt)

@kt=@ fg@kt

: Vk (at; kt) = �EtVk (at+1; kt+1) [fk (kt; nt; zt) + 1� �]

� Resources constrains.

yt + (1� �) kt = ct + kt+1

� We can eliminate the partial di¤erentiations of value function in the F.O.C.and then simplify the equilibrium conditions into 9 equations with 9 endogenousvariables to be determined.

� Equilibrium conditions include F.O.C., de�nitions, and resources constraints.

� yt; ct; kt+1;mt; nt; Rt; �t; zt; ut

uc�ct;mt;1� nt

�= um

�ct;mt;1� nt

�+ �Et

24uc�ct+1;mt+1;1� nt+1

�1 + �t+1

35(1)

ul�ct;mt;1� nt

�= uc

�ct;mt;1� nt

�fn (kt; nt; zt) (2)

uc�ct;mt;1� nt

�= �EtRtuc

�ct+1;mt+1;1� nt+1

�(3)

Rt = 1� � + fk (kt; nt; zt) (4)

yt + (1� �) kt = ct + kt+1 (5)

yt = f (kt; nt; zt) (6)

mt =

1 + �t

1 + �t

!mt�1 (7)

zt = �zt�1 + et (8)

ut = ut�1 + �zt�1 + 't (9)

� Exercises: use Lagrangian to solve the MIU model.

L =1Xt=0

�tu�ct;mt;1� nt

�

+1Xt=0

�t+1Et�t+1

24 f (kt; nt; zt) + (1� �) kt + � t +(1+it�1)bt�1

1+�t+mt�11+�t

�ct � kt+1 �mt � bt

35

� @L@ct; @L@mt

; @L@nt; @L@kt+1

; @L@bt

� Show that combing the F.O.C. of the Lagrangian method gives us the sameequations as by using the Bellman equation.

Example 6: the cash-in-advance (CIA) model

� Walsh (2003), chapter 3, a certainty case

� Utility function:

1Xt=0

�tu (ct)

� CIA constraint:

Ptct �Mt�1 + Tt

� CIA constraint in real terms:

ct �mt�1�t

+ � t

�t �Pt

Pt�1; � t �

Tt

Pt; It = 1 + it

� The budget constraint in real terms:

f (kt) + (1� �) kt + � t +mt�1 + It�1bt�1

�t� ct + kt+1 +mt + bt

wt � f (kt) + (1� �) kt + � t +mt�1 + It�1bt�1

�t

� The budget constraint can be rewritten as:

wt � ct + kt+1 +mt + bt

� Solve the problem

� Two state variables

wt

mt�1

� The Bellman equation is:

V (wt;mt�1) = max [u (ct) + �V (wt+1;mt)]

s:t:

wt � ct + kt+1 +mt + bt

ct �mt�1�t

+ � t

wt = f (kt) + (1� �) kt + � t +mt�1 + It�1bt�1

�t

� To solve the problem, �rst use the de�nition of wt to eliminate wt+1 at the RHSof Bellman equation:

V (wt;mt�1) = max

"u (ct) + �V

f (kt+1) + (1� �) kt+1 + � t+1 +

mt + Itbt

�t+1;mt

!#

� Secondly, from the Lagrangian:

L = max

"u (ct) + �V

f (kt+1) + (1� �) kt+1 + � t+1 +

mt + Itbt

�t+1;mt

!#+�t (wt � ct � kt+1 �mt � bt)

+�t

mt�1�t

+ � t � ct

!

� The �rst order conditions are:

@L

@ct: uc (ct)� �t � �t = 0

@L

@kt+1: �Vw (wt+1;mt) [fk (kt+1) + 1� �]� �t = 0

@L

@bt: �Vw (wt+1;mt)Rt � �t = 0

Rt �It

�t+1

@L

@mt: �Vw (wt+1;mt)

1

�t+1+ �Vm (wt+1;mt)� �t = 0

1

�t+1= Rt �

it

�t+1

� Use envelope theorem to eliminate the partial di¤erentiations of value functionin the �rst order conditions.

� Envelope theorem:

dV

d�=@L

@�� L�

� For details description of envelope theorem, see Dixit, Avinash K. (1990), Opti-mization in Economic Theory, Second Edition, Oxford University Press.

Vw (wt;mt�1) = Lw = �t

Vm (wt;mt�1) = Lm =�t�t

� Rearrange the �rst order conditions as:

uc (ct)� �t � �t = 0 (1)

��t+1 [fk (kt+1) + 1� �]� �t = 0 (2)

��t+1Rt � �t = 0 (3)

Rt �It

�t+1(4)

��t+11

�t+1+ �

�t+1�t+1

� �t = 0 (5)

1

�t+1= Rt �

it

�t+1(6)

� Exercises: use Lagrangian method to solve the CIA model.

L =1Xt=0

�tu (ct)

+1Xt=0

�t�t

0@ f (kt) + (1� �) kt + � t +mt�1+It�1bt�1

�t�ct � kt+1 �mt � bt

1A+

1Xt=0

�t�t

mt�1�t

+ � t � ct

!

References

[1] Uhlig, Harald (1999), "A Toolkit for Analyzing Nonlinear Dynamic StochasticModels Easily," in Ramon Marimon and Andrew Scott (eds.), ComputationalMethods for the Study of Dynamic Economies, Oxford University Press.

[2] Chow, Gregory C. (1997), Dynamic Economics: Optimization by the LagrangeMethod, Oxford University Press, Chapter 2.

[3] Walsh, Carl (2003),Monetary Theory and Policy, Second Edition, The MIT Press,Chapters 2 & 3.