Output Dynamics with Microfoundations

From Solow-Swan to Ramsey-Cass-Koopmans & OLG

Mausumi Das

Lecture 5, EC004, DSE

24 May, 2022

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 1 / 21
Cake Eating Problem: In…nite Horizon
If we extend the time horizon of the cake eating problem to in…nity,
the corresponding dynamic optimization problem will be speci…ed as:
∞

∞
Max. ∑ βt u (ct ) ; u 0 > 0; u 00 < 0;
fct gt =0 ,fW t +1 gt∞=0 t =0
0<β<1

subject to
(i) ct Wt for all t > 0.
(ii) Wt +1 = Wt ct ; ct > 0, Wt +1 > 0 for all t > 0; W0 given.
We will use the dynamic programming technique to solve this in…nite
horizon problem.
We have already discussed the concepts of value function and the
Bellman equation that gives the relationship between the value
functions at two consecutive time periods.
Let’s now apply these concepts to …nd the solution paths of the above
problem.
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 2 / 21
Solving the Cake Eating problem through Dynamic
Programming:

Recall that the Bellman equation for the Cake-eating problem at time
0 is given by:

V (W0 ) Max.u (W0 W1 ) + βV (W1 )

fW 1 g

We now only have to solve the above (reduced form) maximization

problem with respect to W1 .
From the FONC (with respect to W1 ) :

u 0 ( W0 W1 ) = βV 0 (W1 )

Plugging back the value of c0 , the FOC becomes:

u 0 (c0 ) = βV 0 (W0 c0 ) (1)

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 3 / 21
Cake Eating: Dynamic Programming (Contd.)

It seems that from the above FOC we would be able to easily …nd the
optimal c0 once we know the exact speci…cation of u (c ) and the
given value of W0 .
The matter is not that simple though. There is a catch!
Recall the unlike the direct approach, where we had …rst solved for
the optimal consumption stream and then calculated the value
function by plugging in these optimal values in the objective function,
here we have just used the knowledge that such a value function
exists without explicitly deriving it.
Hence we do not know the exact from of V (W0 ) or V (W1 ) or, for
that matter, V 0 (W1 )!
Is there a way to derive the value of V 0 (W1 ) without actually solving
the entire problem?
(If not, then we are back to the direct approach; dynamic
programming approach then will have no special appeal!)
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 4 / 21
Cake Eating: Dynamic Programming (Contd.)

It turns out that we can actually calculate V 0 (W1 ) without solving

the entire problem by applying a standard theorem of optimization,
called the Envelope Theorem.
The Envelope Theorem tells us the following:
Let f (a, x) be a function that is maximized with respect to a set of
variables represented by the vector x (x1 , x2 , ...xn ), where a is a
parameter of the function. Let x (a) represent the optimal solution to
this problem and let f (a, x (a)) represent the corresponding
maximized value of the objective function. Then
the change in the maximal value of the function as the parameter a
changes is captured by the direct impact of the parameter on the
function, holding the value of x …xed at its optimal value, i.e., by the
partial derivative f1 (a, x (a));
The indirect e¤ect, resulting from the change in the optimal value of
x (a) caused by a change in the parameter a, is in fact zero.

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 5 / 21
Cake Eating: Dynamic Programming (Contd.)
Now consider the Bellman equation relevant for the next period (i.e.,
at time period 1, not at t = 0) :
V (W1 ) Max.u (W1 W2 ) + βV (W2 )
fW 2 g

Note that W1 now enters as a parameter in the value function of time

1, while the choice variable here is W2 .
Thus we can apply Envelope theorem in order to measure how
V (W1 ) responds to a change in W1 .
Applying Envelope theorem to the value function of period 1, we get:
V 0 ( W1 ) = u 0 (W1 W2 )
) V 0 ( W 1 ) = u 0 ( c1 ) (2)
Combining the earlier FONC (1) with the above Envelope condition
(2), we get back the familiar Euler equation:
u 0 (c0 ) = βu 0 (c1 )
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 6 / 21
Cake Eating: Dynamic Programming (Contd.)
Since this logical process can be repeated for every two consecutive
time preiods t and t + 1, we can derive the entire series of Euler
equations:
u 0 (ct ) = βu 0 (ct +1 ) for all t = 0, 1, 2, ....
What about the Transversality Condition?
The basic logical reasoning for the transversality condition would
apply here as well; only in the in…nite horizon case there is no
terminal time period. Hence the transversality condition is de…ned as
a limit value.
For the in…nite-horizon cake-eating problem, the Transversality
Condition is written as:
lim βt u 0 (ct ) Wt = 0
t !∞
This is the in…nite horizon counterpart of the …nite horizon TVC
which was given by
βt u 0 (ct ) WT +1 = 0.
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 7 / 21
Cake Eating: Dynamic Programming (Contd.)

Thus we see that using the dynamic programming approach, we can

exactly retrieve the entire optimal consumption path of the agent,
without having to solve the problem directly with a large number of
varibles.
The advantage of dynamic programming approach over the direct
approach lies in reducing the number of variable that we’ll have to
deal with simultaneously. This considerably simpli…es the process.
Having discussed the intuition behind the dynammic programming
approach using the simple cake-eating example, let us now quickly
look at the mathematical framework a bit more formally.

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 8 / 21
A Digression: Dynamic Programming - Generic Case
Consider the following canonical discrete-time, stationary, dynamic
optimization problem:
∞
Max.
∞
fxt +1 gt =0 ,fyt gt∞=0
∑ βt Ũ (xt , yt )
t =0
subject to
(i) yt 2 G̃ (xt ) for all t = 0;
(ii) xt +1 = f˜ (xt , yt ); xt 2 X for all t = 0; x0 given.
Here yt is the control variable (main choice variable); xt is the state
variable (the auxiliary variable which co-moves with the main choice
variable) ; Ũ represents the instantaneous payo¤ function.
(i) speci…es what values the control variable yt is allowed to take (the
feasible set), given the value of xt at time t;
(ii) speci…es evolution of the state variable as a function of previous
period’s state and control variables (state transition equation).
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 9 / 21
Dynamic Programming : Generic Case (Contd.)

It is often convenient to use the state transition equation given by (ii)

to eliminate the control variable and write the dynamic programming
problem in terms of the state variable alone:
∞
Max.
∞ ∑ βt U (xt , xt +1 )
f x t +1 g t =0 t = 0

subject to
(i) xt +1 2 G (xt ) for all t = 0; x0 given.
∞
Let xt +1 t =0
denote the corresponding solution .
Then we write the value function of the above problem at time 0 as a
function of x0 :
∞
V (x0 ) Max.
∞ ∑ βt U (xt , xt +1 ) ;
f x t +1 g t =0 t = 0
xt +1 2 G (xt ) for all t = 0;

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 10 / 21
Dynamic Programming : General Case (Contd.)

Using the same logic as before, we can write the Bellman equation of
the above problem at time 0 in the following way:

V (x0 ) = Max [U (x0 , x1 ) + βV (x1 )] ; x0 given.

x 1 2G (x 0 )

As before, the Bellman equation breaks down the ini…nite horizon

dynamic optimization problem into a two-stage problem:
Given x0 , what is the optimal value of x1 ;
what is the optimal continuation path (V (x1 )).
Thus it reduces the initial optimization problem to a simple
optimization exercise involving only one variable (x1 ).

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 11 / 21
Dynamic Programming : Generic Case (Contd.)

Likewise, we can write down the Bellman equation for any two
consecutive time periods t and t + 1 as:

V (xt ) = Max [U (xt , xt +1 ) + βV (xt +1 )] for all t.

x t +1 2 G (x t )

Or equivalently:

V (x ) = Max [U (x, x̃ ) + βV (x̃ )] for all x 2 X . (3)

x̃ 2G (x )

where x and x̃ denote the values of the state variable in any two
consecutive time periods.
The maximizer of the right hand side of equation above is called a
policy function:
x̃ = π (x ),
which solves the RHS of the Bellman Equation above.
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 12 / 21
Dynamic Programming : Generic Case (Contd.)

If we knew the exact form of the value function V (.) and were it
di¤erentiable, we could have easily found the policy function by
solving the following FONC (the Euler Equation):

∂U (x, x̃ )
x̃ : + βV 0 (x̃ ) = 0. (4)
∂x̃

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 13 / 21
Dynamic Programming : Generic Case (Contd.)

Of course the value function is not known to us apriori.

In fact we do not even know whether it exists; if yes then
whether it is unique, whether it is continuous, whether it is
di¤erentiable etc.
A lot of theorems in Dynamic Programming go into establishing
conditions under which a value function exists, is unique and has all
the nice properties (continuity, di¤erentibility and others).
For now, without going into futher details, we shall simply assume
that all these conditions are satis…ed for all the economic examples we
shall encouter in this course..
In other words, we shall assume that for our problems the value
function exists and is well-behaved (even though we do not know
its precise form).

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 14 / 21
Dynamic Programming : Generic Case (Contd.)

Once the existence of the value function is established (here, by

assumption), we can then solve the FOC above (the Euler Equation)
to get the policy function.
As before we can also …nd the derivative function V 0 (x̃ ) by
augmenting the Bellman equation by one period and then
appying the Envelope Theorem to the augmented Bellman
equation.

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 15 / 21
Dynamic Programming : Generic Case (Contd.)

Since the Bellman equation is de…ned for all x 2 X , we therefore get

a similar relationship between x̃ and its subsequent state value (x̂):

V (x̃ ) = Max [U (x̃, x̂ ) + βV (x̂ )] .

x̂ 2G (x̃ )

Then applying Envelope Theorem:

∂U (x̃, x̂ )
V 0 (x̃ ) = . (5)
∂x̃
Combining the Euler Equation and the Envelope Condition, we get
the following equation:

∂U (x, x̃ ) ∂U (x̃, x̂ )
+β =0
∂x̃ ∂x̃

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 16 / 21
Dynamic Programming : Generic Case (Contd.)
Replacing x, x̃, x̂ by their suitable time subscripts:
∂U (xt , xt +1 ) ∂U (xt +1 , xt +2 )
+β = 0; x0 given. (6)
∂xt +1 ∂xt +1
Equation (6) is a di¤erence equation which we should be able to solve
to derive the time path of the state variable xt .
Notice that (6) is a di¤erence equation of order 2. To solve this
equation, we need two boundary conditions.
One boundary condition is speci…ed by the given initial value x0 .
But we need another boundary condition to precisely pin down the
solution path.
Typically in a Dynamic Programming problem such a boundary
condition is provided by the following Transversality condition
(TVC):
∂U (xt , xt +1 )
lim βt xt = 0. (7)
t !∞ ∂xt
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 17 / 21
Transversality Condition and its Interpretation:

The TVC is again to be interpretated as a complementary slackness

condition in the following way:
∂U (xt ,x t +1 )
as t ! ∞, if βt ∂x t > 0, then xt = 0;
∂U (xt ,xt +1 )
on the other hand, as t ! ∞, if xt > 0, then βt ∂x t =0
t t +1∂U (x ,x )
In interpreting the TVC, notice that ∂xt captures the marginal
increment in the pay-o¤ function associated with an increase in the
current stock, or its shadow price.
The TVC states that if (present discounted value of) the shadow
price is positive then at the terminal date, agents will not leave any
stock unused (i.e., would not leave any postive stock at the end of
the period); on the other hand, if any stock indeed remains unused at
the terminal date, then it must be the case that its shadow valuation
is zero.

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 18 / 21
Dynamic Programming - Generic Case: Reference

Interested students can look up D. Acemoglu (2009): Introduction

to Modern Economic Growth, Chapter 6, for the generic dynamic
programming problem, associated theorems and proofs.
(I will NOT ask for any of the theorems or proofs in this course.)

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 19 / 21
Back to Household’s Choice Problem: In…nite Horizon
Let us now go back to optimization problem of household h under
in…nite horizon given by:
∞

∞
Max. ∞ ∑ βt u cth ; u 0 > 0; u 00 < 0
fcth gt =0 ,fath+1 gt =0 t =0

subject to
(i) cth 5 wt + rt ath + (1 δ)ath for all t = 0;
(ii) ath+1 = wt + (1 + rt δ)ath cth ; ath = 0 for all t = 0; a0h given.
Notice that this is not a "stationary" dynamic programming problem
because the wage rate (wt ) and the rental rate (rt ) keep changing
over time, which means the nature of the state transition equation
will keep changing over time.
You could still use the dynamic programming technique developed
earlier (with some caveats). But let’s …rst solve a stationary
counterpart of the problem by assuming that wt and rt remain
unchanged at some given values: w and r .
Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 20 / 21
Back to Household’s Choice Problem: In…nite Horizon, No
Borrowing

Then the "stationary" optimization problem becomes:

∞

∞
Max. ∞ ∑ βt u cth ; u 0 > 0; u 00 < 0
fcth gt =0 ,fath+1 gt =0 t =0

subject to

(i) cth 5 w + rath + (1 δ)ath for all t = 0;

(ii) ath+1 = w + (1 + r δ)ath cth for all t = 0; a0h given.

Exercise: Write down the corresponding Bellman equation between

time 0 and 1, apply the envelope condition to the value function at
time 1, and derive the correspnding Euler equation.

Das (Lecture 5, EC004, DSE) Solow to RCK & OLG 24 May, 2022 21 / 21