ECON408: Computational Methods in Macroeconomics

Stochastic Dynamics, AR(1) Processes, and Ergodicity

Jesse Perla
jesse.perla@ubc.ca
University of British Columbia

1 / 72
Table of contents
Overview
Random Variables Review
Stochastic Processes
AR(1) Processes
Stationary Distributions
Nonlinear Stochastic Processes
Stochastic Growth Model

2 / 72
Overview

3 / 72
Motivation and Materials
In this lecture, we will introduce our stochastic processes and review
probability
Our first example of a stochastic process is the AR(1) process (i.e. auto-
regressive of order one)
→ This is a simple, univariate process, but it is directly useful in many cases
We will also introduce the concept of ergodicity to help us understand long-
run behavior
While this section is not directly introducing new economic models, it provides
the backbone for our analysis of the wealth and income distribution

4 / 72
Deterministic Processes
We have seen deterministic processes in previous lectures, e.g. the linear

X t+1 = aX t + b

→ These are coupled with an initial condition X , which enables us to see

0

the evolution of a variable

→ The state variable, X , could be a vector
t

→ The evolution could be non-linear X t+1 , etc.

= h(X t )

But many states in the real world involve randomness

5 / 72
Materials
Adapted from QuantEcon lectures coauthored with John Stachurski and
Thomas J. Sargent
→ AR1 Processes
→ LLN and CLT
→ Continuous State Markov Chains
1 using LaTeXStrings, LinearAlgebra, Plots, Statistics
2 using Random, StatsPlots, Distributions, NLsolve
3 using Plots.PlotMeasures
4 default(;legendfontsize=16, linewidth=2, tickfontsize=12,
5 bottom_margin=15mm)

6 / 72
Random Variables Review

7 / 72
Random Variables
Random variables are a collection of values with associated probabilities
For example, a random variable Y could be the outcome of a coin flip
→ Let Y = 1 if heads and Y = 0 if tails
→ Assign probabilities P(Y = 1) = P(Y = 0) = 0.5

or a normal random variable with mean μ and variance σ , denoted 2

2
(y−μ)
Y ∼ N (μ, σ )
2
has density p(y) =
1
exp (−
2σ
2
)
√ 2πσ 2

8 / 72
Discrete vs. Continuous Variables
If discrete (e.g., X ∈ {x 1 , … , x N } ) , then
→ The probability mass function (pmf) is the probability of each value
N
p ∈ R

→ Such that ∑ , and p

N
pi = 1 i ≥ 0
i=1

→ i.e. p i = P(X = x i )

If continuous, then the probability density function (pdf) is the probability of

each value and can be represented by a function
→ p : R → R if X is defined on R
∞
→ ∫
−∞
p(x)dx = 1 , and p(x) ≥ 0

b
→ P(X = a) = 0 in our examples, and P(X ∈ [a, b]) = ∫
a
p(x)dx

9 / 72
Normal Random Variables
1 mu = 0.1
2 sigma = 0.5
3 d = Normal(0.1, sigma) # SD not variance
4 x = range(mu - 2 * sigma,
5 mu + 2 * sigma;
6 length=100)
7 plot(x, pdf.(d, x); label="Normal PDF",
8 xlabel="x", ylabel=L"p(x)",
9 size=(600,400))

10 / 72
Comparing to a Histogram
1 n = 1000
2 x_draws = rand(d, n) # gets n samples
3 histogram(x_draws; label="Histogram",
4 xlabel="x", ylabel=L"\hat{p}(x)",
5 normalize=true, size=(600,400))
6 plot!(x, pdf.(d, x); label="Normal PDF",
7 lw=3)

11 / 72
Normal Random Variables
Normal random variables are special for many reasons (e.g., central limit
theorems)
They are the only continuous random variable with finite variance that is
closed under linear combinations
→ For independent X ∼ N (μ X , σ
2
X
) and Y ∼ N (μ Y , σ
2
Y
)

→ aX + bY ∼ N (aμ X + bμ Y , a σ
2 2
X
2
+ b σ
2
Y
)

→ Also true with multivariate normal distributions

Common transformation taking out mean and variance
→ Could draw Y ∼ N (μ, σ )
2

→ Or could draw X ∼ N (0, 1) and then Y = μ + σX

12 / 72
Expectations
For discrete-valued random variables

E[f (X)] = ∑ f (x i )p i

i=1

For continuous valued random variables

∞

E[f (X)] = ∫ f (x)p(x)dx

−∞

13 / 72
Moments
The mean of a random variable is the first moment, E[X]
The variance of a random variable is the second moment, E[(X − E[X]) 2
]

→ Note the recentering by the mean. Could also calculate as

2 2
E[X ] − E[X]

Normal random variables are characterized by their first 2 moments

14 / 72
Law(s) of Large Numbers
Let X , X , … be independent and identically distributed (iid) random
1 2

variables with mean μ ≡ E(X) < ∞, then let

n
1
X̄ n ≡ ∑ Xi
n
i=1

One version is Kolmogorov’s Strong Law of Large Numbers

P ( lim X̄ n = μ) = 1
n→∞

→ i.e. the average of the random variables converges to the mean

15 / 72
Sampling and Plotting the Mean
1 function ksl(distribution, n = 100)
2 title = nameof(typeof(distribution))
3 observations = rand(distribution, n)
4 sample_means = cumsum(observations) ./ (1:n)
5 mu = mean(distribution)
6 plot(repeat((1:n)', 2), [zeros(1, n); observations']; title, xlabel="n",
7 label = "", color = :grey, alpha = 0.5)
8 plot!(1:n, observations; color = :grey, markershape = :circle,
9 alpha = 0.5, label = "", linewidth = 0)
10 if !isnan(mu)
11 hline!([mu], color = :black, linewidth = 1.5, linestyle = :dash,
12 grid = false, label = L"\mathbb{E}[X]")
13 end
14 return plot!(1:n, sample_means, linewidth = 3, alpha = 0.6, color = :green, label = L"\bar{X}_n")
15 end
ksl (generic function with 2 methods)

16 / 72
LLN with the Normal Distribution
1 dist = Normal(0.0, 1.0) # unit normal
2 ksl(dist)

17 / 72
LLN with the Exponential
f (x) =
α
1
exp(−x/α) for x ≥ 0 with mean α
1 dist = Exponential(0.2)
2 ksl(dist)

18 / 72
LLN with the Cauchy?
f (x) = 1/(π(1 + x ))
2
, with median = 0 and E(X) undefined
1 Random.seed!(0); # reproducible results
2 dist = Cauchy() # Doesn't have an expectation!
3 sample_mean = cumsum(rand(dist, n)) ./ (1:n)
4 plot(1:n, sample_mean, color = :red, alpha = 0.6, label = L"\bar{X}_n",
5 xlabel = "n", linewidth = 3)
6 hline!([0], color = :black, linestyle = :dash, label = "", grid = false)

19 / 72
Monte-Carlo Calculation of Expectations
One application of this is the numerical calculation of expectations
Let X be a random variable with density p(x), and hence
∞
f (x)p(x)dx (or ∑ f (x )p if discrete)
N
E[f (X)] = ∫ i i
−∞ i=1

These integrals are often difficult to calculate analytically, but if we can draw
X ∼ p, then we can approximate the expectation by

n
1
E[f (X)] ≈ ∑ f (x i )
n
i=1

Then by the LLN this converges to the true expectation as n → ∞

20 / 72
Discrete Example
Let X be a discrete random variable with N states and probabilities p i

Then E[f (X)]

N
= ∑ f (x i )p i
i=1

For example, the Binomial distribution and f (x) = log(x + 1)

1 # number of trials and probability of success (mean(dist), dot(vals, p)) = (5.0, 5.000000000000008)
2 dist = Binomial(10, 0.5)
3 plot(dist;label="Binomial PMF",
4 size=(600,400))
5 vals = support(dist) # i.e. 0:10
6 p = pdf.(dist, vals)
7 # Calulate the expectation manually
8 @show mean(dist), dot(vals, p);

21 / 72
Using Monte-Carlo
1 N = 500 (f_expec, f_expec_mc) = (1.7526393207741702,
1.7552834928857293)
2 # expectation with PMF, then MC
3 f_expec = dot(log.(vals .+ 1), p)
4 x_draws = rand(dist, N)
5 f_x_draws = log.(x_draws .+ 1)
6 f_expec_mc = sum(f_x_draws) / N
7 @show f_expec, f_expec_mc
8 # Just calculate sums then divide by N
9 f_means = cumsum(f_x_draws)./(1:N)
10 plot(1:length(f_means), f_means;
11 label=L"\frac{1}{n}\sum_{i=1}^n f(x_i)",
12 xlabel="n", ylabel=L"\bar{f}(X)",
13 size=(600,400))
14 hline!([f_expec];linestyle = :dash,
15 label = L"\mathbb{E}[f(X)]")

22 / 72
Stochastic Processes

23 / 72
Stochastic Processes
A stochastic process is a sequence of random variables
→ We will focus on discrete time stochastic processes, where the sequence
is indexed by t = 0, 1, 2, …
→ Could be discrete or continuous random variables
Skipping through some formality, assume that they share the same values but
probabilities may change
Denote then as a sequence {X t}
∞
t=0

24 / 72
Joint, Marginal, and Conditional Distributions
Can ask questions on the probability distributions of the process
The joint distribution of {X ∞
t } t=0 or a subset
→ In many cases things will be correlated over time or else no need to be a
process
The marginal distribution of X for any t t

→ This is a proper PDF, marginalized from the joint distribution of all values
Conditional distributions, fixing some values
→ e.g. X t+1 given X t, X t−1 , etc. are known

25 / 72
Markov Process
Before we go further, lets discuss a broader class of these processes useful in
economics
A Markov process is a stochastic process where the conditional distribution
of X t+1given X , X , … is the same as the conditional distribution of
t t−1

X t+1given X t

→ i.e. the future is independent of the past given the present

Note that with the AR(1) model, if I know X then I can calculate the PDF of
t

X t+1 directly without knowing the past

This is “first-order” since only one lag is required, but could be higher order
→ A finite number of lags can always be added to the state vector to make it
first-order

26 / 72
AR(1) Processes

27 / 72
A Simple Auto-Regressive Process with One Lag
X t+1 = aX t + b + cW t+1

Just added randomness to the deterministic process from time t to t + 1

W t+1 ∼ N (0, 1) is IID “shocks” or “noise”
Could have an initial condition for X Or could have an initial distribution
0

→ Xt is a random variable, and so can X 0

→ “Degenerate random variable” if P (X 0 = x) = 1 for some x

→ Assume X 0 , and v
∼ N (μ 0 , v 0 ) 0 → 0 is the degenerate case

28 / 72
Evolution of the AR(1) Process
Both W t+1 and X are assumed to be normally distributed
0

As we discussed, linear combinations of normal random variables are normal

→ So X is normal for all t by induction
t

Furthermore, we have a formula for the recursion

→ If X t ∼ N (μ t , v t ) , then X t+1
2 2
∼ N (aμ t + b, a v t + c )

→ Hence, the evolution of the mean and variance follow a simple difference
equation μ = aμ + b and v
t+1 t = a v + c t+1
2
t
2

→ Let X t ∼ ψ t ≡ N (μ t , v t )

29 / 72
Visualizing the AR(1) Process
1 a = 0.9
2 b = 0.1
3 c = 0.5
4
5 # initial conditions mu_0, v_0
6 mu = -3.0
7 v = 0.6
0.6

30 / 72
Visualizing the AR(1) Process
1 sim_length = 5
2 x_grid = range(-5, 7, length = 120)
3
4 plt = plot(;size = (600, 400))
5 for t in 1:sim_length
6 mu = a * mu + b
7 v = a^2 * v + c^2
8 dist = Normal(mu, sqrt(v))
9 plot!(plt, x_grid, pdf.(dist, x_grid),
10 label = L"\psi_{%$t}", linealpha = 0.7)
11 end
12 plt

31 / 72
From a Degenerate Initial condition
Cannot plot ψ since it is a point mass at μ
0 0

1 mu = -3.0
2 v = 0.0
3 plt = plot(;size = (600, 400))
4 for t in 1:sim_length
5 mu = a * mu + b
6 v = a^2 * v + c^2
7 dist = Normal(mu, sqrt(v))
8 plot!(plt, x_grid, pdf.(dist, x_grid),
9 label = L"\psi_{%$t}", linealpha = 0.7)
10 end
11 plt

32 / 72
Practice with Iteration
Let us practice creating a map and iterating it
We will need to modify our iterate_map function to work with vectors
Let x ,
⊤
≡ [μ v]

1 function iterate_map(f, x0, T)

2 x = zeros(length(x0), T + 1)
3 x[:, 1] = x0
4 for t in 2:(T + 1)
5 x[:, t] = f(x[:, t - 1])
6 end
7 return x
8 end
iterate_map (generic function with 1 method)

33 / 72
Implementation of the Recurrence for the AR(1)
1 function f(x;a, b, c)
2 mu = x[1]
3 v = x[2]
4 return [a * mu + b, a^2 * v + c^2]
5 end
6 x_0 = [-3.0, 0.6]
7 T = 5
8 x = iterate_map(x -> f(x; a, b, c), x_0, T)
2×6 Matrix{Float64}:
-3.0 -2.6 -2.24 -1.916 -1.6244 -1.36196
0.6 0.736 0.84616 0.93539 1.00767 1.06621

34 / 72
 Using Matrices
a 0 b
x t+1 = [ ]x t + [ ]
2 2
0 a c



      
≡A ≡B

1 A = [a 0; 0 a^2]
2 B = [b; c^2]
3 x = iterate_map(x -> A * x + B, x_0, T)
2×6 Matrix{Float64}:
-3.0 -2.6 -2.24 -1.916 -1.6244 -1.36196
0.6 0.736 0.84616 0.93539 1.00767 1.06621

35 / 72
Fixed Point?
Whenever you have maps, you can ask whether a fixed point exists
This is especially easy to check here. Solve,
→ μ = aμ + b ⟹ μ =
b

1−a

→ 2
v = a v + c
2
⟹ v =
1−a
c
2

Lets check for a fixed point numerically

1 sol = fixedpoint(x -> A * x + B, x_0)
2 @show sol.zero
3 @show b/(1-a), c^2/(1-a^2);
sol.zero = [1.0000000000000266, 1.3157894736842035]
(b / (1 - a), c ^ 2 / (1 - a ^ 2)) = (1.0000000000000002, 1.3157894736842108)

36 / 72
Existence of a Fixed Point
The important of a is also clear when we look at the A matrix
We know the eigenvalues of a diagonal matrix are the diagonal elements
→ i.e., λ 1 = a and λ 2 = a
2

If |a| < 1, then a < 2

|a| < 1 and hence the maxim absolute value of the
eigenvalues below 1
As we saw in the univariate case, conditions of this sort were crucial to
determine whether the systems would converge
We will see more complicated versions of the A matrix as we move into richer
“state space models”

37 / 72
Evolution of the Probability Distributions
1 x_0 = [-1.0, 0.1] # tight
2 T = 10
3 f(x) = A * x + B
4 x = iterate_map(f, x_0, T)
5 x_star = fixedpoint(f, x_0).zero
6 plt = plot(Normal(x_star[1], sqrt(x_star[2]));
7 label = L"\psi^*",
8 style = :dash,
9 size = (600, 400))
10 for t in 1:T
11 dist = Normal(x[1, t], sqrt(x[2, t]))
12 plot!(plt, dist, label = L"\psi_{%$(t-1)}",
13 linealpha = 0.7)
14 end
15 plt

38 / 72
Stationary Distributions

39 / 72
Fixed Points and Steady States
Recall in the lecture on deterministic dynamics that we discussed fixed point
and steady states x = f (x ) has a fixed point x if x = f (x )
t+1 t
∗ ∗ ∗

→ e.g. x t+1 = ax t + b has x ∗

=
b

1−a
if |a| < 1

We can also interpret as a steady state x as lim ∗

t→∞ xt = x
∗
for some x 0

→ Stability looked at stability which told us about which x the process ∗

would approach from points x near x 0

∗

The key: for x if we apply f (x

∗ ∗
) evolution equation and remain at that point

40 / 72
Stationary Distributions
Analogously, with stochastic processes we can think about applying the
evolution equation to random variables
→ Instead of a point, we have a distribution ψ ∗

→ Then rather than checking x = f (x ), we check ψ ∼ f (ψ ), where

∗ ∗ ∗ ∗

that notation is loosely taking into account the distribution of shocks

Similar to stability, we can consider if repeatedly applying f (⋅) repeatedly to
various ψ converges to ψ
0
∗

41 / 72
AR(1) Example
Take X t+1 = aX t + b + cW t+1 if |a| < 1 for W t+1 ∼ N (0, 1)

Recall If X t ∼ N (μ t , v t ) ≡ ψ t , then using properties of Normals

→ X t+1 ∼ N (aμ t + b, a v t + c ) ≡ ψ t+1
2 2

→ We derived the fixed point of the mean and variance iteration as

2
∗ ∗ ∗ b c
ψ ∼ N (μ , v ) = N ( , 2
)
1−a 1−a

Apply the evolution equation to ψ we demonstrate that ψ∗ ∗

∼ f (ψ )
∗

2 2
b 2
c 2
b c
N (a + b, a + c ) = N ( , )
2 2
1 − a 1 − a 1 − a 1 − a

→ i.e., from any initial condition, the distribution of X converges to ψ t

∗

42 / 72
What if a > 1 ?
1 a,b,c = 1.1, 0.2, 0.25
2 A = [a 0; 0 a^2]
3 B = [b; c^2]
4 f(x) = A * x + B
5 T = 15
6 x = iterate_map(f, [0.0, 0.1], T)
7 plt = plot(Normal(x[1, end], sqrt(x[2, end]));
8 label = L"\psi_{%$T}",
9 size = (600, 400))
10 for t in 1:5
11 dist = Normal(x[1, t], sqrt(x[2, t]))
12 plot!(plt, dist, label=L"\psi_{%$(t-1)}",
13 linealpha = 0.7)
14 end
15 plt

43 / 72
Analyzing the Failure of Convergence
2

If it exists, the stationary distribution would need to be ψ ∗

≡ N (
b

1−a
,
c

1−a
2
)

Note that if b > 0 we get the drift of the process forward

→ But, just as in the case of the deterministic process, this just acts as a
force to move the distribution, not spread it out
In fact, with b = 0 the mean of ψ is always 0, but the variance grows without
t

bound if c > 0
Lets plot the a = 1, b = 0 case

44 / 72
What if a = 1, b = 0 ?
1 a,b,c = 1.0, 0.0, 0.25
2 A = [a 0; 0 a^2]
3 B = [b; c^2]
4 f(x) = A * x + B
5 T = 15
6 x = iterate_map(f, [0.0, 0.1], T)
7 plt = plot(Normal(x[1, end], sqrt(x[2, end]));
8 label = L"\psi_{%$T}",
9 size = (600, 400))
10 for t in 1:5
11 dist = Normal(x[1, t], sqrt(x[2, t]))
12 plot!(plt, dist, label=L"\psi_{%$(t-1)}",
13 linealpha = 0.7)
14 end
15 plt

45 / 72
Ergodicity
There are many different variations and definitions of ergodicity
Among other things, this rules out are cases where the process is “trapped” in
a subset of the state space and can’t swith out
Also ensures that the distribution doesn’t spread or drift asymptotically
Ergodicity lets us apply LLNs to the stochastic process, even though they are
not independent

46 / 72
Ergodicity
We will consider a process {X t}
∞
t=0
with a stationary distribution ψ ∗

The process is ergodic if for any f : R → R (with regularity conditions)

T
1
∗
lim ∑ f (X t ) = ∫ f (x)ψ (x)dx
T →∞ T
t=1

→ i.e. the time average of the function converges to the expectation of the
function. Mean ergodic if only require this to work for f (x) = x

47 / 72
Iteration with IID Noise
Adapt scalar iteration for iid noise
1 function iterate_map_iid(f, dist, x0, T)
2 x = zeros(T + 1)
3 x[1] = x0
4 for t in 2:(T + 1)
5 x[t] = f(x[t - 1], rand(dist))
6 end
7 return x
8 end
9 a,b,c = 0.9, 0.1, 0.05
10 x_0 = 0.5
11 T = 5
12 h(x, W) = a * x + b + c * W # iterate given random shock
13 x = iterate_map_iid(h, Normal(), x_0, T)
6-element Vector{Float64}:
0.5
0.5252717486805177
0.5306225876900339
0.46819901566492783
0.532032538532688
0.583020976850554
48 / 72
Demonstration of Ergodicity with Mean
1 T = 2000
2 x_0 = 0.5
3 x = iterate_map_iid(h, Normal(), x_0, T)
4 x_means = cumsum(x)./(1:(T+1))
5 plot(0:T, x_means;
6 label=L"\frac{1}{t}\sum_{s=0}^{t-1} X_s",
7 xlabel = "t", size = (600, 400))
8 hline!([b/(1-a)], color = :black,
9 linestyle = :dash,
10 label = L"\mathbb{E}[X]")

49 / 72
Starting at the Stationary Distribution
A reasonable place to start many simulations is a draw from the stationary
distribution

1 Random.seed!(20)
2 x_0 = rand(Normal(b/(1-a), sqrt(c^2/(1-a^2))))
3 x = iterate_map_iid(h, Normal(), x_0, T)
4 x_means = cumsum(x)./(1:(T+1))
5 plot(0:T, x_means;
6 label=L"\frac{1}{t}\sum_{s=0}^{t-1} X_s",
7 xlabel = "t", size = (600, 400))
8 hline!([b/(1-a)], color = :black,
9 linestyle = :dash,
10 label = L"\mathbb{E}[X]")

50 / 72
The Speed of Convergence
The speed with which the process converges towards its stationary
distribution is important
Key things which govern this transition will be
→ Autocorrelation: As a goes closer to 0, the faster it converges back
towards the mean - as with deterministic processes
→ Variances: Wth large c the noise may dominate and the ψ becomes
∗

broader

51 / 72
Close to a Random Walk
1 Random.seed!(20)
2 a,b,c = 0.99, 0.01, 0.05
3 h(x, W) = a * x + b + c * W
4 T = 2000
5 x_0 = 0.5
6 x = iterate_map_iid(h, Normal(), x_0, T)
7 x_means = cumsum(x)./(1:(T+1))
8 plot(0:T, x_means;
9 label=L"\frac{1}{t}\sum_{s=0}^{t-1} X_s",
10 xlabel = "t", size = (600, 400))
11 hline!([b/(1-a)], color = :black,
12 linestyle = :dash,
13 label = L"\mathbb{E}[X]")

52 / 72
Dependence on Initial Condition
Intuition: ergodicity is that the initial conditions “wear off” over time
However, even if a process is ergodic and has a well-defined stationary
distribution, it may take a long time to converge to it
This is very important in many quantitative models:
→ How much does your initial wealth matter for your long-run?
→ If your wages start low due to discrimination, migration, or just bad luck,
how long does it converge?
→ If we provide subsidies to new firms, how long would it take for that to
affect the distribution of firms?

53 / 72
Example of a Non-Ergodic Stochastic Process
Between t = 0 and t = 1 a coin is flipped (e.g., result of key exam)
→ If heads: income follows X t+1 = aX t + b + cW t+1 with b = 0.1 for
t ≥ 1

→ If tails: income follows X t+1 = aX t + b + cW t+1 with b = 1.0 for t ≥ 1

The initial condition and early sequence cannot be forgotten

If there is ANY probability of switching between careers, then it is ergodic
because it “mixes”

54 / 72
Moving Average Representation, MA(∞), for AR(1)
From X t = aX t−1 + b + cW t , iterate backwards to X and W
0 1

X t = a (aX t−2 + b + cW t−1 ) + b + cW t

2
= a X t−2 + b(1 + a) + c(W t + aW t−1 )

2
= a (aX t−3 + b + cW t−2 ) + b(1 + a) + c(W t + aW t−1 )

t−1 t−1

t j j
= a X0 + b ∑ a + c ∑ a W t−j

j=0 j=0

t t−1
1 − a
t j
= a X0 + b + c ∑ a W t−j
1 − a
j=0

55 / 72
Interpreting the Auto-Regressive Parameter
The distribution of X then depends on the distribution of X and the
t 0

distribution of the sum of t − 1 iid random variables

If X and W are normal, then X is normal since it is a linear combination
0 t t

t t−1
1 − a
t j
Xt = a X0 + b + c ∑ a W t−j
1 − a
j=0

→ If a = 1 then the initial condition is never “forgotten”

→ If a = 1, W shocks are just as important determining the distribution
t−j

of X because the a doesn’t “decay” over time

t
2

56 / 72
Simulation of Moving Average Representation
1 X_0 = 0.5 # degenerate prior
2 a, b, c = 0.9, 0.1, 0.05
3 A = [a 0; 0 a^2]
4 B = [b; c^2]
5 T = 20
6 num_samples = 10000
7 Xs = iterate_map(x -> A * x + B, [X_0, 0], T)
8 X_T = Normal(Xs[1, end], sqrt(Xs[2, end]))
9 W = randn(num_samples, T)
10 # Comprehensions and generators example, looks like math
11 X_T_samples = [a^T * X_0 + b * (1-a^T)/(1-a) + c * sum(a^j * W[i, T-j] for j in 0:T-1)
12 for i in 1:num_samples]
13 histogram(X_T_samples; xlabel="x", normalize=true,
14 label=L"Samples of $X_{%$T}$ using MA($\infty$)")
15 plot!(X_T; label=L"Analytic $X_{%$T}$", lw=3)

57 / 72
Simulation of Moving Average Representation

58 / 72
Nonlinear Stochastic Processes

59 / 72
Nonlinearity with Additive Shocks
A useful class involves nonlinear functions for the drift and variance

X t+1 = μ(X t ) + σ(X t )W t+1

→ IID W t+1 with E[W t+1 ] = 0 and frequently E[W 2

t+1
] = 1

Nests our AR(1) process

→ μ(x) = ax + b and σ(x) = c

60 / 72
Auto-Regressive Conditional Heteroskedasticity (ARCH)
For example, we may find that time-series data has time-varying volatility and
depends on 1 lags

X t+1 = aX t + σ t W t+1

→ And that the variance increases as we move away from the mean of the
stationary distribution σ = β + γX
2
t t
2

Hence the process becomes an ARCH(1)

1/2
2
X t+1 = aX t + (β + γX ) W t+1
t

61 / 72
Simulation of ARCH(1)
1 a = 0.7
2 beta, gamma = 5, 0.5
3 X_0 = 1.0
4 T = 200
5 h(x, W) = a * x + sqrt(beta + gamma * x^2) * W
6 x = iterate_map_iid(h, Normal(), X_0, T)
7 plot(x; label = L"X_t", size = (600, 400))

62 / 72
AR(1) with a Barrier
Nonlinearity in economics often comes in various forms of barriers,
e.g. borrowing constraints
Consider our AR(1) except that the process can never go below 0

X t+1 = max{aX t + b + cW t+1 , 0.0}

We could stop the process at this point, but instead we will continue to iterate

63 / 72
Simulation of AR(1) with a Barrier
1 a,b,c = 0.95, 0.00, 0.05
2 X_min = 0.0
3 h(x, W) = max(a * x + b + c * W, X_min)
4 T = 1000
5 x_0 = 0.5
6 x = iterate_map_iid(h, Normal(), x_0, T)
7 plot(x; label = L"X_t", size = (600, 400))

64 / 72
Histogram of the AR(1) with a Barrier
There isn’t a true density of ψ due to the point mass at 0
∗

1 T = 20000
2 x = iterate_map_iid(h, Normal(), x_0, T)
3 histogram(x; label = L"X_t", normalize = true,
4 xlabel = "x", size = (600, 400))

65 / 72
Stochastic Growth Model

66 / 72
Simple Growth Model with Stochastic Productivity
Turning off population growth, for f (k) = k
α
, and s, δ constants

k t+1 = (1 − δ)k t + sZ t f (k t ), given k 0

Let log productivity, z t ≡ log Z t , follow an AR(1) process (why logs?)

log Z t+1 = a log Z t + b + cW t+1

67 / 72
Stationary Distribution of Productivity
2

Recall that the stationary distribution of log Z is N

t (
b

1−a
,
c

1−a
2
)

Given the stationary distribution of Z is lognormal, we can check ergodicity

t

1 a, b, c = 0.9, 0.1, 0.05

2 Z_0 = 1.0
3 T = 20000
4 h(z, W) = a * z + b + c * W
5 z = iterate_map_iid(h, Normal(), log(Z_0), T)
6 Z = exp.(z)
7 histogram(Z; label = L"Z_t", normalize = true,
8 xlabel = "Z", size = (600, 400))
9 plot!(LogNormal(b/(1-a), sqrt(c^2/(1-a^2))),
10 lw = 3, label = L"\psi^*")

68 / 72
Practice with Iteration and Multivariate Functions
1 function iterate_map_iid_vec(h, dist, x0, T)
2 x = zeros(length(x0), T + 1)
3 x[:, 1] = x0
4 for t in 2:(T + 1)
5 # accepts whatever type rand(dist) returns
6 x[:, t] = h(x[:, t - 1], rand(dist))
7 end
8 return x
9 end
iterate_map_iid_vec (generic function with 1 method)

69 / 72
Simulation of the Stochastic Growth Model
1 alpha, delta, s = 0.3, 0.1, 0.2
2 a, b, c = 0.9, 0.1, 0.1
3 function h(x, W)
4 k = x[1]
5 z = x[2]
6 return [(1-delta) * k + s * exp(z) * k^alpha,
7 a * z + b + c * W]
8 end
9 x_0 = [7.0, log(2.0)] # k_0, z_0
10 T = 150
11 x = iterate_map_iid_vec(h, Normal(), x_0, T)
12 plot(x[1, :]; label = L"k_t", xlabel = "t", size = (600, 400), legend=:topright)
13 plot!(exp.(x[2, :]), label = L"Z_t")
14 dist = LogNormal(b/(1-a), sqrt(c^2/(1-a^2)))
15 hline!([mean(dist)]; linestyle = :dash, label = L"\mathbb{E}[Z_t]")
16 hline!([quantile(dist, 0.05)]; lw=0, fillrange = [quantile(dist, 0.95)], fillalpha=0.2, label = "5/95")

70 / 72
Simulation of the Stochastic Growth Model

71 / 72
Ergodicity and Capital Accumulation
Evaluate the closed-form steady-state capital k for the deterministic model
∗

1 # Remember nonstochastic steady-state

2 k_ss_det= (s*mean(dist)/delta)^(1/(1-alpha))
3
4 T = 200000
5 x = iterate_map_iid_vec(h, Normal(), x_0, T)
6 histogram(x[1, :]; label = L"k_t",
7 normalize = true, xlabel = "k",
8 alpha=0.5, size = (600, 400))
9 vline!([k_ss_det]; linestyle = :dash, lw=3,
10 label = L"k_{ss}(c = 0)")

72 / 72