Stochastic growth

Martin Ellison

1 Motivation
In this lecture we apply the techniques of dynamic programming to real
macroeconomic problems. We use the technique of value function iterations
to derive the solution of the stochastic growth model, in which a representa-
tive agent makes optimal choices between current consumption and investing
in a productive asset, capital. To fully understand the intuition behind our
results and methods, we begin by analysing a simple non-stochastic growth
model and presenting Gauss codes to solve the model numerically by com-
puter. The stochastic version of the growth model is presented in the latter
half of the lecture. In this case, a simple stochastic structure is sucient to
demonstrate the theoretical concepts and numerical simulation.

2 Key reading
The majority of this lecture is derived from chapter 5 of Dynamic Eco-
nomics: Quantitative Methods and Applications by Jrme Adda and Rus-
sell Cooper, Massachusetts Institute of Technology, 2003.

1
3 Other reading
The stochastic growth model and its solution are covered in almost all ad-
vanced textbooks in macroeconomics. The relevant chapter of Recursive
Macroeconomic Theory, 2nd ed by Lars Ljungqvist and Tom Sargent, MIT
Press, 2000, is number 11.The analytical result with full depreciation is well
known from Dynamic Macroeconomic Theory by Tom Sargent, Harvard
University Press, 1987.

4 Non-stochastic growth model

In the non-stochastic growth model, the problem of the representative agent
is to allocate resources between consumption ct and investment in capital kt ,
as in the continuous time Ramsey model. Capital is completely malleable,
being able to be transformed into consumption at a rate of one-to-one. Since
the model is deterministic, there is no uncertainty. Consumption gives in-
stantaneous utility through the function u(ct ), which is strictly increasing
and strictly concave. Capital is productive according to the production func-
tion yt = f (kt ), which is similarly strictly increasing and strictly concave.
Capital stored depreciates at the rate when used in production. With the
representative agent discounting future utility at the rate , the problem is
as follows:

P

max t1 u(ct )
{ct } t=1

s:t:
yt = f (kt )
yt = ct + it
kt+1 = kt (1 ) + it

2
 where it is gross investment. Taking our now familiar dynamic program-
ming approach, we can write the value function as

V (k) = max [u(f (k) + (1 )k k 0 ) + V (k 0 )]

0k

The first order condition is

u0 (c) = V 0 (k 0 )

As in the previous lecture, we do not know the form is the value function
V (k), but we do know its first derivative V 0 (k). Dierentiating from the
definition of the value function

V 0 (k) = u0 (c) [f 0 (k) + (1 )]

Rolling forward one period

V 0 (k 0 ) = u0 (c0 ) [f 0 (k0 ) + (1 )]

Substitute in the first order condition

u0 (c) = u0 (c0 ) [f 0 (k 0 ) + (1 )]

This is the Euler equation for optimal consumption smoothing in the non-
stochastic growth model, analogous to the Euler equation in the cake-eating
example of the previous lecture. It is economically intuitive. the left hand
side shows the utility benefit of increasing consumption by one unit in the
current period. The right hand side shows the benefit of postponing con-
sumption to the next period. One unit of consumption postponed increases
the productive capital next period by one unit, and so generates extra output
f 0 (k0 ). However, the extra capital depreciates during production so that the
net additional consumption available is f 0 (k0 ) +(1 ). This is then weighted

3
by the marginal utility of consumption to arrive at the utility benefit of the
extra consumption. It is discounted by because the extra consumption
only occurs in the next period.
To proceed from the value function and first order conditions, we have
three options. Firstly, it may be possible to solve the problem analytically by
guessing the form of the value function or policy function. Secondly, we could
linearise or log-linearise the first order conditions to obtain an approximate
solution. Thirdly, we can apply value function iterations to derive a numer-
ical solution. In this lecture, we will investigate the first and third options.
The second option, approximation, is beyond the scope of the this course,
although its is not conceptually dicult. Interested readers are referred to
Recursive Methods for Computing Equilibria of Business Cycle Models by
Gary Hansen and Ed Prescott in Frontiers of Business Cycle Research,
edited by Tom Cooley, PUP, 1995. The book Computational Methods for
the Study of Dynamic Economies, edited by Ramon Marimon and Andrew
Scott, CUP, 1999, is also a good reference.

5 An analytically tractable non-stochastic growth

model example
In general, it is not possible to solve analytically for the value function and
policy function. However, it is possible under very strict assumptions. We
begin by defining functional forms for the utility function and production
function.

u(c) = ln c
f (k) = k

These functional forms are fairly standard, although the production func-

4
tion does preclude the use of labour as a factor of production. Our final as-
sumption is even less realistic. We assume = 1 so that capital depreciation
occurs at the rate of 100%. In other words, any capital saved to the next
periods depreciates completely after production. The intertemporal budget
constraint is therefore k0 = i rather that k 0 = k(1 ) + i. We begin our
solution by making a guess at the form of the value function.

V (k) = A + B ln k

This is an informed, rather than random, guess because we know that

the state variable is k and both the utility and production functions are
logarithmic. We proceed to confirm this is correct by showing that, if this is
the correct form of the value function, optimising behaviour implies a value
function of the same form. If the value function is correct, the first order
condition u0 (c) = V 0 (k 0 ) implies

1 B
=
c k
The budget constraint and 100% depreciation define

k = c + k 0

Solving these equations, we can obtain expressions for the policy functions
of consumption and future capital.

1
c = k
1 + B

B
k =
0
k
1 + B

We now return to the value function and show that these policy functions
are consistent with a value function of the form originally posited.

5

1 B
V (k) = A + B ln k = ln k + A + B ln k
1 + B 1 + B

Collecting terms in the constant and ln k gives two equations which de-
termine A and B.

1 B
A = ln + A + B ln
1 + B 1 + B
B = + B

This is an example of the method of undetermined coecients. The

second equation defines B = =(1 ), which can be used in the first
equation to calculate A. Once B is defined, the policy functions become

k 0 = k
c = (1 )k

In this simple example, the policy functions take a particularly neat form.
In each period, the consumer saves a constant proportion of output k
and consumes a constant proportion (1 ). The higher the parameters
and the more the consumer saves. This makes intuitive sense. A higher
implies the consumer places more weight on future periods and so the return
to saving increases. Similarly, higher improves the productivity of capital
and so promotes saving.
The analytical results obtained in this section are elegant and intuitive.
However, they are derived under very restrictive assumptions. More gener-
ally, it will be dicult to guess the intuitive form of the value function. In
most cases it will be impossible.

6
6 Numerical non-stochastic growth model ex-
ample
Our numerical example maintains the same function form for the production
function but uses the more general CRRA utility function with coecient
equal to the coecient of relative risk aversion.

c1
u(c) =
1
f (k) = k

In the limit of 1, the utility function becomes logarithmic as before.

In contrast to the analytical example, we allow depreciation to be less than
100% so we have a full set of dynamic equations showing how state variables
evolve.

y = c+i
k0 = k(1 ) + i

The numerical code below solves for repeated iterations of the value func-
tion.

T (W )(k) = max [u(f (k) + (1 )k k 0 ) + W (k 0 )]

0k
We do not check Blackwells suciency conditions for a contraction map-
ping since monotonicity and discounting hold by the same arguments ex-
pressed in the previous lecture. The computer program is written in Matlab
and the code complimentary is available from the homepage of the Adda and
Cooper book at

http://www.eco.utexas.edu/~cooper/dynprog/dynprog1.html

7
 There are four main parts to the code: parameter value initialisation,
state space definitions, value function iterations and output.
Start new program by clearing the work space. Parameters are initialised
with beta the discount factor, sigma the coecient of relative risk aversion,
alpha the exponent on capital in the production function and delta the de-
preciation rate.

CLEAR;
beta=0.9;
sigma=1;
alpha=0.75;
delta=0.3;

We now construct a grid of possible capital stocks. The grid is centred

on the steady-state capital stock k , which is obtained from the steady-state
value of the Euler equation of the stochastic growth model. We have

u0 (c ) = u0 (c ) [f 0 (k ) + (1 )]

With f 0 (k ) = k 1 , this reduces to

1 = k 1 + (1 )

Solving for k .

1
1
1 1
k =



The grid is centred around this point, with maximum (khi) and minimum
(klo) values 110% and 90% of the steady-state value. There are N discrete
points in the grid.

8
 N=100;
kstar=(((1/(alpha*beta))-((1-delta)/alpha)))^(1/(alpha-1));
klo=Kstar*0.9;
khi=Kstar*1.1;
step=(khi-klo)/N;
k=klo:step:khi;
n=length(k);

Initial values for the value function are taken by assuming that all capital
is always consumed immediately. In this case, consumption is equal to the
product of capital k plus its depreciated value (1 )k. Before we calculate
the value function we trasform our variables in a Matlab friendly format. s
and s1 are a matrices with every column being k and k respectively. The
syntax : defines element-by-element operation, so each element of a vector
or matrix is risen to the power of the exponent.

kalpha=k.^alpha;
colones=ones(n,1);
s = colones*kalpha;
s1 = colones*k;
ytot = s+(1-delta)*s1 ;

The IF ... THEN ... ELSE structure applies the correct utility function.
Obviously, these starting values will be incorrect but repeated value function
iterations will converge to the true value function.

IF sigma==1
v=log(ytot);
ELSE
v=ytot.^(1-sigma)/(1-sigma);
END

9
 For use in the value function iterations, we define an n n matrix C,
which is indexed by column by the current capital stock k and by row by
the future capital stock k0 . Each element of C contains the necessary level of
consumption at capital stock k to achieve capital stock k0 in the future. The
elements are obtained by summing current production k and depreciated
capital (1 )k and then subtracting future consumption k0 . The matrix is
particularly useful since it is invariant across value function iterations: the
choice of c to achieve k0 given k does not depend on the value function.
We therefore do not need to recalculate this matrix at each value function
iteration. The final part of this section of code maps consumption levels C
into utility levels U by applying the suitable utility function. The syntax :=
defines element-by-element operation rather than matrix division implied by
using the = operator.

rowones= colones;
I = k*rowones;
J= colones*k;
C = (J.^alpha)-I +(1-delta)*J;
IF sigma==1
U=log(C);
ELSE
U=(C.^(1-sigma))/(1-sigma);
END

Now we can use our first guess of the value function to get the second
estimate v1 which will be the start of our next t iterations.

r=U+beta*v;
v1=max(r);

10
 For each iteration, the nn matrix w1 collects the consequences of dier-
ent actions at dierent initial capital levels. Each column of w corresponds
to a given initial capital stock level k. The rows of w1 then show the value
of leaving dierent future capital stocks k0 . The value of each choice is ob-
tained by summing the value of the implied consumption level (from the
utility matrix U ) and the value of carrying forward capital k 0 to the next
period (from the value function v1 derived in the previous iteration). Once
the consequences of dierent actions have been collected in w1, the value
function iteration proceeds by selecting the maximum value form each of its
columns. As these values are implicitly stored as a row vector we line them
up vertically into a matrix w. This matrix becomes the next iteration of the
value function.

t=100
FOR j=1:t
w=ones(n,1)*v1;
w1=U+beta*w;
v1=max(w1);
END

Value function iterations are now complete. The final value function is
stored in V al and the indices of the future k 0 choices are stored in ind. These
latter indices are converted into k 0 values in optk.

[val,ind]=max(w1);
optk = K(ind);

To illustrate the results, it is useful to plot how k 0 varies as a function of

k. This is the policy function in Figure 5.1 of Adda and Cooper.

figure(1)

11
 plot(K,[optk K],LineWidth,2)
xlabel(K);
ylabel(Optimal Future Capital);
legend(Policy function,45 degree line,4);
text(0.4,0.65,45 degree line,FontSize,18)
text(0.4,0.13,Optimal Future Capital,FontSize,18)

12.5

12

11.5
K`

11

10.5

10

Policy function
45 degree line
9.5
9.5 10 10.5 11 11.5 12 12.5
K

The policy function is upward-sloping and crosses the 45o line at the
steady-state capital stock. When capital is below its steady state k < k ,
capital rises and k0 > k. Conversely, when k > k capital falls and k 0 < k.
This ensures convergence. Note also that the relationship between k and
k0 appears roughly linear, which might have been expected given that the
departure from the guaranteed linearity of the analytical example is not large.

12
A second useful figure is Figure 5.2 from Adda and Cooper, showing net
investment k 0 k as a function of k. The figure confirms that net investment
is positive when k < k and negative when k > k .

figure (2)
plot(K,(optk-k),LineWidth,2)
xlabel(K);
ylabel(K - K);

0.15

0.1

0.05

0
K` - K

-0.05

-0.1

-0.15

-0.2
9.5 10 10.5 11 11.5 12 12.5
K

To gain an insight into the dynamics of the adjustment process, it is

useful to calculate the path of convergence of capital if initial capital is not
equal to its steady-state value. The exercise is similar to the tracing out of
impulse response functions in a structural vector autoregression analysis. p

13
is the number of periods over which to follow convergence. The indices of
the capital choices are stored in the p 1 vector mi and the capital stocks
themselves are stored in the p 1 vector m. The initial index is taken as 1
so the initial capital stocks is at its maximum value.

p=50;
mi=zeros(p,1);
m=zeros(p,1);
mi(1)=N;
m(1)=K(mi(1));
FOR i=2:p
mi(i)=ind(mi(i-1));
m(i) = K(mi(i));
END
t=1:1:50
figure(3)
plot(t,m)
xlabel(t);
ylabel(k(t));

14
 12.2

12

11.8
k(t)

11.6

11.4

11.2

11
0 5 10 15 20 25 30 35 40 45 50
t

The figure shows that convergence to k is not instantaneous. The half-life

is approximately 10 periods.

7 Stochastic growth model

The non-stochastic growth model helps in understanding the role of the value
and policy functions but any realistic application in macroeconomics requires
models to be stochastic. This is particularly true if we have a long-term
ambition to bring macroeconomic models to the data. The concepts and
techniques involved in handling stochastic models are no more complex than
the deterministic case, although state spaces tend to expand as the values
of stochastic shocks enter the state space. This eventually leads to problems
known as the curse of dimensionality. If each state in a 4-dimensional
model is discretised into 10 points then the total number of grid points rises
to 10 10 10 10 = 10000. For this reason, many stochastic models are

15
kept to a small scale.
Our stochastic version of the growth model incorporates shocks to pro-
duction technology that enter multiplicatively in the production function.
Commonly known as the Solow residual, At accounts for unexplained changes
in productivity. It changes the problem of the representative agent to be

P

max t1 u(ct )
{ct } t=1

s:t:
yt = At f (kt )
yt = ct + it
kt+1 = kt (1 ) + it

Without loss of generality, we restrict our attention to stochastic processes

for At that are Markovian, i.e. At is a sucient statistic to form the expec-
tation of At+1 . In dynamic programming form, the value function is defined
by the Bellman equation.

V (A; k) = max u(Af (k) + (1 )k k 0
) + EA0 |A V (A ; k )
0 0
0k

Compared with the non-stochastic growth model, there are two changes.
Firstly, the value function has two arguments: the current level of technology
A and the current capital stock k. Secondly, the continuation part of the
value function is only defined in terms of an expectation. It is the expected
continuation value, where the expectation is evaluated across the distribution
of future technology A0 given current technology A. The expectations remains
in the first order condition.

u0 (c) = EA0 |A [u0 (c0 )(A0 f(k 0 ) + (1 ))]

16
 It is possible to solve this problem analytically as before for the limiting
case of 100% depreciation and logarithmic utility. Such an exercise is left for
the reader. Instead, we jump straight to a numerical solution of a stochastic
growth model through value function iterations.

8 Numerical stochastic growth model exam-

ple
To maintain comparability with the non-stochastic growth model example,
we use the same function forms for the utility and production functions.

c1
u(c) =
1
f (k) = Ak

We introduce a stochastic element to A in the simplest possible manner.

We assume technology can either be high A or low A with equal probability,

i.e.

A0 = A with probability 0.5

A0 = A with probability 0.5

As a more general case, one could imagine having a range of discrete
values for A0 with dierent probabilities. In this way, the distribution of A0
could be matched, say, to a normal distribution. The way we have defined
the stochastic element means that the distribution of A0 is independent of A.
Economically, this is equivalent to saying that technology shocks only have
a temporary and not permanent or persistent eects. It is simple (although
not trivial) to extend the computer code to handle persistent shocks.

17
 The main dierence to the computer code is that policy functions and
value functions become n 2 matrices, with the columns corresponding to
policy and value functions conditional on high or low current productivity.
We can write out the expectation in the value function as follows:

0
V (A; k) = max u(Af (k) + (1 )k k ) +
0
V (A; k ) + V (A; k )
0
k0 2

This Bellman equation forms the basis of the value function iterations.
In practical terms, for each iteration we use the matrices CH and CL for
the consumption levels necessary to achieve future capital k 0 when current
capital is k and current technology is high and low respectively. U H and UL
are the associated utility levels.

CLEAR ALL
sigma=1;
beta=0.9;
alpha=.75;
delta=0.3;
AH=1;
AL=0.99;
N=100;
kstarH=(((1/(AH*alpha*beta))-((1-delta)/(AH*alpha))))^(1/(alpha-
1));
kstarL=(((1/(AL*alpha*beta))-((1-delta)/(AL*alpha))))^(1/(alpha-
1));
kstar=0.5*(KstarH+KstarL);
klo=kstar*0.9;
khi=kstar*1.1;
step=(khi-klo)/N;

18
k=klo:step:khi;
n=length(k);
kalpha=k.^alpha;
colones=ones(n,1);
s = colones*kalpha;
s1 = colones*K;
ytotH = AH*s+(1-delta)*s1;
ytotL = AL*s+(1-delta)*s1;
IF sigma==1
vH=log(ytotH);
vL=log(ytotL);
ELSE
vH=ytotH.^(1-sigma)/(1-sigma);
vL=ytotL.^(1-sigma)/(1-sigma);
END
rowones= colones;
I = K*rowones;
J= colones*K;
Jalpha=J.^alpha;
CH = (AH*Jalpha)-I +(1-delta)*J;
CL = (AL*Jalpha)-I +(1-delta)*J;
IF sigma==1
UH=log(CH);
UL=log(CL);
ELSE
UH=(CH.^(1-sigma))/(1-sigma);
UL=(CL.^(1-sigma))/(1-sigma);
END
rH=UH+beta*0.5*(vH+vL);

19
 rL=UL+beta*0.5*(vH+vL);
vH1=max(rH);
vL1=max(rL);
t=100
FOR j=1:t
w=ones(n,1)*(vH1+vL1);
wH1=UH+beta*0.5*w;
wL1=UL+beta*0.5*w;
vH1=max(wH1);
vL1=max(wL1);
END
[valH,indH]=max(wH1);
[valL,indL]=max(wL1);
optkh = K(indH);
optkl = K(indL);

FIGURE(1)
plot(K,[K optkh optkl],LineWidth,2)
xlabel(K);
ylabel(K);
legend(45 degree line,Policy function- High Tech,Policy function-
Low Tech,4);

FIGURE(2)
plot(K,[(optkh-K) (optkl-K)],LineWidth,2)
xlabel(K);
ylabel(K- K);

20
 12

45 degree line

11.5

11
Policy function- Low Tech
Policy function- High Tech
K`

10.5

10

9.5
9.5 10 10.5 11 11.5 12
K

0.15

0.1 High Technology

0.05

0
K`- K

Low Technology
-0.05

-0.1

-0.15

-0.2
9.5 10 10.5 11 11.5 12
K

The policy function figures now have two lines, each corresponding to
policy conditional on current productivity. In the figures, the higher line is
for high technology and the lower line is for low technology. The dynamics

21
of the system will change according to the value of A. The system is globally
stable since for very low capital stocks the capital stock is always increasing
and for very high capital stocks the capital stock is always decreasing. For
intermediate levels of the capital stock, the system converges in distribution
if not to a fixed point.
To appreciate the dynamics of the system, we present a simple simulation
of the evolution of the capital stock over p periods. Initial capital stock is
taken to be at the average steady-state value. The function RAN D(1; 1)
draws a single random number from the uniform distribution [0; 1]. By com-
paring its value with 0.5, we are able to draw shocks from the stochastic
distribution of A. A typical stochastic simulation is shown below. We also
print out the standard deviations of output, consumption and investment.

p=500;
kt=Kstar*ones(p,1);
kti=(N/2)*ones(p,1);
FOR i=2:p
IF RAND(1,1)>0.5
yt(i)=AH*(kt(i-1))^alpha;
kti(i)=indH(kti(i-1));
kt(i)=K(kti(i));
ELSE
yt(i)=AL*(kt(i-1))^alpha;
kti(i)=indL(kti(i-1));
kt(i)=K(kti(i));
END
ct(i)=yt(i)+(1-delta)*kt(i-1)-kt(i);
it(i)=yt(i)-ct(i);
END

22
 t=1:1:500;
figure (3)
plot(t,kt)
xlabel(t);
ylabel(k(t));

syt=std(yt(10:p));
sct=std(ct(10:p));
si=std(it(10:p));

11.05

11

10.95

10.9
k(t)

10.85

10.8

10.75

10.7
0 50 100 150 200 250 300 350 400 450 500
t

Variable SD
y 0.041
c 0.015
i 0.029

23
 Even with such a simple example, the simulation results are highly infor-
mative. Consumption is by far the least volatile variable, reflecting strong
incentives for consumption smoothing in the model. Analysis of more com-
plex models in this way has been a bedrock of Real Business Cycle research
since the seminal contribution of Kydland and Prescott in Econometrica in
1982.

24