ECON60522 Applied Macroeconometrics

Univariate Time Series Processes

Arthur Sinko

University of Manchester

Semester 2, Spring 2024

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 1/34
ECON61001 Revision

↭ ECON61001 Econometric Methods lecture Time Series Data

introduced univariate processes
↭ Key concepts include:
↭ (weak) stationarity
↭ white noise process
↭ trend stationarity

↭ Types of processes:
↭ autoregressive (AR) models
↭ moving average (MA) models
↭ autoregressive-moving average (ARMA) models

↭ Will not repeat all this in ECON60522 lectures!

↭ included in notes on univariate processes for completeness

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 2/34
Lag Operator
↭ Lag operator simplifies much time series notation
↭ Define: lag operator L such that
Lyt = yt →1
and, by repetition,
Lk yt = L....L yt = yt →k
k times

↭ ARMA(p, q )
yt = δ + φ1 yt →1 + ... + φp yt →p + ε t → ϱ1 ε t →1 → ... → ϱq ε t →q
for white noise ε t , is written compactly as
φ ( L ) yt = δ + ϱ (L) ε t
φ(L) = 1 → φ1 L → ... → φp Lp , ϱ (L) = 1 → ϱ1 L → ... → ϱq Lq
↭ Note minus sign in MA definition, rather than plus in
ECON61001
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 3/34
Stationary AR(1) Processes
Unconditional Mean and Variance

↭ AR (1)
yt = δ + φ1 yt →1 + ε t .
Here and throughout ε t is white noise with Var [ε t ] = σ2 .
↭ Stationarity condition: |φ1 | < 1.
↭ Stationary AR (1) has (ECON61001):

δ
E [ yt ] = µ=
1 → φ1
σ2
Var [yt ] = γ0 =
1 → φ12

↭ Stationarity ↑ µ & δ have the same sign (since 1 → φ1 > 0)

and also γ0 > 0.

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 4/34
Stationary AR(1) Processes
MA Representation

↭ AR (1) is (1 → φL)yt = δ + ε t
↭ MA representation of AR (1):

yt = (1 → φL)→1 [δ + ε t ]
δ
= + (1 + φL + φ2 L2 + ...)ε t
1→φ
= µ + ε t + φε t →1 + φ2 ε t →2 + ...

Note: (1 → φL)→1 = 1 + φL + φ2 L2 + ... for stationary AR (1)

[Exercise Sheet 1].

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 5/34
Stationary AR(1) Processes
Autocovariances I
↭ Using δ = (1 → φ1 )µ, AR (1) is

yt → δ = φyt →1 + ε t
yt → µ = φ { yt → 1 → µ } + ε t
↭ Definition: autocovariance at lag j:

γj = E [yt → µ][yt →j → µ]

Substituting on RHS,

γj = E [φ{yt →1 → µ} + ε t ][yt →j → µ]
= φE [yt →1 → µ][yt →j → µ] + E [ε t (yt →j → µ)]
= φγj →1 j >0

since yt →j → µ = ε t →j + φε t →j →1 + φ2 ε t →j →2 + ... is
uncorrelated with ε t .
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 6/34
Stationary AR(1) Processes
Autocovariances II
↭ As

γj = φγj →1 = φ2 γj →2
= φj γ0 j = 0, 1, 2...
φj σ 2
= j = 0, 1, 2...
1 → φ2
↭ Result established in lecture notes by direct use of MA
representation.

↭ Also

γj = E [yt → µ][yt →j → µ] = E [yt →j → µ][yt → µ]

= E [yt → µ][yt +j → µ] by stationarity
= γ→ j j = 1, 2...

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 7/34
Stationary AR(1) Processes
Autocorrelations
↭ Definition: autocorrelation at lag j:
γj
ρj =
γ0
analogous to usual definition of correlation:
Cov [x, y ]
Corr [x, y ] = !
Var [x ] ↓ Var [y ]
↭ For AR (1) :
γj φj γ0
ρj = =
γ0 γ0
= φj j = 0, 1, 2, ...
ρj = ρ →j j = →1, →2, ...
↭ Either monotonically decreasing for j = 0, 1, 2, ... (0 < φ < 1)
or alternating in sign & decreasing in magnitude (→1 < φ < 0)
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 8/34
Stationary AR(1) Processes
Theoretical Autocorrelation Examples

Theoretical ACF
1.0

phi = 0.7
phi = −0.7
0.5
acf2

0.0
−0.5

0 5 10 15 20

lag

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 9/34
Random Walk Processes
↭ Random walk with drift is

yt = δ + yt → 1 + ε t

and when δ = 0 this a random walk.

↭ Important in financial econometrics (stock market prices,
exchange rates, etc)
↭ Process is nonstationary AR (1) with φ = 1

↭ Repeated substitution yields

yt = ( yt → 2 + δ + ε t → 1 ) + δ + ε t
= yt →2 + 2δ + ε t + ε t →1
t
= y0 + δt + ∑ ε i
i =1

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 10/34
Random Walk Processes
Forecasting
↭ Driftless random walk satisfies
∆yt = yt → yt →1 = ε t
Hence, given Yt = {yt , yt →1 , yt →2 , ...}
E [∆yt +m |Yt ] = 0, m = 1, 2, 3...
the m-step ahead forecast of yt +m is
E [yt +m |Yt ] = y"t +m|t = yt
↭ Expected accuracy is measured by mean-square forecast error
MSFE (m ) = E [y"t +m|t → yt +m ]2 .
↭ For the random walk
# $2
m
MSFE (m ) = E ∑ ε
i =1 t +i
= mσ2
increases linearly as m ↔ ∞.
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 11/34
Random Walk Processes
Mean and Variance
↭ Using
t
yt = y0 + δt + ∑ ε i
i =1
↭ Mean:
E [yt ] = E [y0 ] + δt
depends on t if δ ↗= 0.
↭ Variance:
Var [yt ] = E [yt → E (yt )]2
= E [{y0 → E (y0 )} + ε 1 + ε 2 + ... + ε t ]2
= Var [y0 ] + tσ2
since y0 is uncorrelated with yi (i > 0) & ε t is white noise.
↭ Hence Var [yt ] grows with time and
Var [yt ] ↔ ∞ as t

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 12/34
AR(p) Processes

↭ AR(p) process is

yt = δ + φ1 yt →1 + φ2 yt →2 + ... + φp yt →p + ε t

↭ Using lag operator

φ ( L ) yt = δ + ε t

with lag polynomial

φ(L) = 1 → φ1 L → φ2 L2 → ... → φp Lp

↭ Stationarity more complicated than AR(1) case

↭ Condition ∑pi=1 |φi | < 1 is NOT correct

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 13/34
AR(p) Processes
Stationarity Condition I
↭ Stationarity condition for AR(p)

yt = δ + φ1 yt →1 + φ2 yt →2 + ... + φp yt →p + ε t

is the stability condition for p th order di!erence equation

xt = φ1 xt →1 + φ2 xt →2 + ... + φp xt →p

or xt → φ1 xt →1 → φ2 xt →2 → ... → φp xt →p = 0.
↭ Roots of the characteristic equation

z p → φ1 z p →1 → φ2 z p →2 → ... → φp →1 z → φp = 0

(z → λ1 )(z → λ2 )...(z → λp ) = 0
↭ require all |λj | < 1, j = 1, ..., p

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 14/34
AR(p) Processes
Stationarity Condition II
↭ Characteristic equation of AR(p):

(z → λ1 )(z → λ2 )...(z → λp ) = 0

↭ Alternatively, AR polynomial

φ (L) = 1 → φ1 L → φ2 L2 → ... → φp Lp
= (1 → λ1 L)(1 → λ2 L)...(1 → λp L) = 0

↑ roots
→1
1 → λj L = 0 ↔ L = λj
↭ Hence AR(p) stationarity condition is equivalently:
↭ all roots of characteristic equation |λj | < 1
% %
% →1 %
↭ all roots of AR polynomial %λj % > 1 (”lie outside the unit
circle”)
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 15/34
AR(p) Processes
Stationarity Condition Examples I
↭ AR(2) process
yt = 1.3yt →1 → 0.4yt →2 + ε t
has characteristic equation
z 2 → 1.3z + 0.4 = (z → 0.8)(z → 0.5) = 0
with roots z = 0.8, 0.5. Hence the process is stationary.
↭ Equivalently, factorising the lag polynomial,
φ (L) = 1 → 1.3L + 0.4L2
= (1 → 0.8L)(1 → 0.5L)
the coe”cients |λi | < 1, i = 1, 2 and the process is stationary.
In this form, the roots are
1 1
L= = 1.25 and L = =2
0.8 0.5
which are larger than one (in absolute value).
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 16/34
AR(p) Processes
Stationarity Condition Examples II
↭ AR(2) process

yt = 0.9yt →1 + 0.9yt →2 + ε t

has characteristic equation

z 2 → 0.9z → 0.9 = (z + 0.6)(z → 1.5) = 0

with roots z = →0.6, 1.5. Since |λ2 | > 1, the process is not
stationary.
Factorising the lag polynomial leads to the same conlcusion,
with

φ (L) = 1 → 0.9L → 0.9L2

= (1 + 0.6L)(1 → 1.5L).

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 17/34
Stationary AR(p) Processes
Mean
↭ Stationary AR(p):
φ ( L ) yt = δ + ε t
Since E [yt ] = E [yt →1 ] = ... = E [yt →p ]

φ ( L ) E [ yt ] = δ + E [ε t ] = δ
δ
µ=
φ (1)

where φ(1) = 1 → φ1 → φ2 → ... → φp [replace L by 1 in φ(L)]

↭ Since
φ(1) = (1 → λ1 )(1 → λ2 )...(1 → λp )
with |λj | < 1, j = 1, .., p, then φ(1) > 0.
↭ µ and δ must have have the same sign
↭ δ = 0 if and only if µ = 0
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 18/34
Stationary AR(p) Processes
MA Representation

↭ Stationary AR(p) process

φ ( L ) yt = δ + ε t

can be inverted to obtain the MA representation

δ
yt = + φ (L ) →1 ε t
φ (1)
= µ + ε t → ϱ1 ε t →1 → ϱ2 ε t →2 → ...

of infinite order
↭ MA coe”cients ϱj (j = 1, 2, ...) are functions of φ1 , φ2 , ..., φp
(only)

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 19/34
Stationary AR(p) Processes
Autocovariances and Autocorrelations
↭ Using δ = µ φ(1), AR(p) can be written
φ(L){yt → µ} = ε t
Multiply by yt →j → µ & taking E , autocovariances satisfy
γj = φ1 γj →1 + φ2 γj →2 + ... + φp γj →p j = 1, 2, ...
γ0 = φ1 γ→1 + φ2 γ→2 + ... + φp γ→p + σ2
[Derivation of general case not examinable for ECON60552,
but special case in Exercise Sheet 1.]
↭ Autocorrelations satisfy the Yule-Walker equations:
γj /γ0 = φ1 (γj →1 /γ0 ) + φ2 (γj →2 /γ0 ) + ... + φp (γj →p /γ0 )
ρj = φ1 ρj →1 + φ2 ρj →2 + ... + φp ρj →p j = 1, 2, ...
↭ Wide range of possible patterns (see notes)
↭ ρj ↔ 0 as j ↔ ∞ for all stationary processes
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 20/34
Stationary AR(p) Processes
Example

↭ Stationary AR(2):

yt = 0.1 + 1.3yt →1 → 0.4yt →2 + ε t

↭ Mean
0.1
µ = E [ yt ] = =1
1 → 1.3 + 0.4
↭ Autocorrelations

ρj = 1.3ρj →1 → 0.4ρj →2 j = 1, 2, ...

ρ1 = 1.3ρ0 → 0.4ρ→1 = 1.3 → 0.4ρ1 ↑ ρ1 = 0.93
ρ2 = 1.3ρ1 → 0.4ρ0 = 1.3 ↓ 0.93 → 0.4 = 0.81
ρ3 = 1.3ρ2 → 0.4ρ1 = 0.68 etc

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 21/34
Stationary AR(p) Processes
Forecasting
↭ Principles are precisely as for AR(1)
↭ One-step ahead:

y"t +1|t = δ + φ1 yt + φ2 yt →1 + ... + φp yt →p +1

or, more generally,

y"t +m|t = δ + φ1 y"t +m→1|t + φ2 y"t +m→2|t + ... + φp y"t +m→p |t

where y"t +j |t = yt +j when j ↘ 0

↭ As for the AR(1)

y"t +m|t ↔ µ = E [yt ] as m ↔ ∞

MSFE (m ) ↔ γ0 = Var [yt ] as m ↔ ∞

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 22/34
Stationary AR(p) Processes
Forecasting Example

↭ Stationary AR(2):

yt = 0.1 + 1.3yt →1 → 0.4yt →2 + ε t

↭ If y99 = 4.5 & y100 = 5.2

y"t +m|t = yt +m→1|t → 0.4"

0.1 + 1.3" yt + m → 2 | t
y"101|100 = 0.1 + 1.3y100 → 0.4y99
= 0.1 + 1.3 ↓ 5.2 → 0.4 ↓ 4.5 = 5.06
y"102|100 = y101|100 → 0.4y100
0.1 + 1.3"
= 0.1 + 1.3 ↓ 5.06 → 0.4 ↓ 5.2 = 4.60
y"103|100 = y102|100 → 0.4"
0.1 + 1.3" y101|100 = 4.05

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 23/34
ARMA Processes

↭ ARMA(p, q) process

φ ( L ) yt = δ + ϱ (L) ε t
φ (L) = 1 → φ1 L... → φp Lp
ϱ (L) = 1 → ϱ1 L → ... → ϱq Lq

↭ Stationary if roots of φ(L) are > 1 in absolute value

↭ ”Invertible” if roots of ϱ (L) are > 1 in absolute value

↭ A stationary and invertible ARMA has:

↭ infinite order MA representation yt = δ
+ φ (L ) →1 ϱ (L ) ε t
φ (1)

↭ infinite order AR representation ϱ (L)→1 φ(L)yt = δ

+ εt
ϱ (1)

↭ AR approximation widely used in macroeconometrics!

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 24/34
Parsimony of ARMA(p,q) processes
Issues with ARMA specifications. Assume that the true process is:

yt = ) t

However, we can represent it as ARMA(1,1):

(1 → φL)yt = (1 → φL))t

yt = φyt →1 + )t → φ)t →1
or more general ARMA(p,p):

φ ( L ) yt = φ ( L ) ) t
p p
yt = )t + ∑ φi yt →i → ∑ φi )t →i
i =1 i =1

It is not as unusual as you can think!

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 25/34
Issues with ARMA estimation Monte Carlo simulation.
library(lmtest)
library(forecast)
#library(car)
set.seed(1)
x = rnorm(1000)
out = arima(x,order = c(1,0,1))
coeftest(out)

##
## z test of coefficients:
##
## Estimate Std. Error z value Pr(>|z|)
## ar1 0.6827 0.1698 4.02 5.8e-05 ***
## ma1 -0.7339 0.1574 -4.66 3.1e-06 ***
## intercept -0.0113 0.0274 -0.41 0.68
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

R2 = 1 - out$sigma2/var(x)
print(paste("R2 = ",round(R2,4)))

## [1] "R2 = 0.0059"

#linearHypothesis(out,"ar1 + ma1 = 0")

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 26/34
Kuznets Filter

Simon Kuznets got a Nobel Prize in 1970 “for his empirically

founded interpretation of economic growth which has led to new
and deepened insight into the economic and social structure and
process of development”. In his work (1930) he studied yearly data
for various US industries and detected 15-25 years infrastructural
investment cycles. To smooth the jumpy data he was adopting the
following procedure:
1. xt≃ = ∑2i =→2 xt →i
2. xt≃≃ = xt≃+5 → xt≃→5
The only problem is that the cycles of similar frequency can be
found even if there is no dependence exists in the original data!

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 27/34
Kuznets filter. Monte-Carlo sumulation.
set.seed(123)
x = rnorm(T <- 10000)
#require(forecast)
y <- z <- x*0
for(i in 3:(T-2)) {
y[i] = sum(x[(i-2):(i+2)])/5}

for(i in 6:(T-5)) {
z[i] = y[i+5] - y[i-5]}
Acf(z,lag.max = 30,main = "ACF of smoothed series",ylim=c(-0.6,0.6))

ACF of smoothed series

0.6
0.2
ACF

−0.2
−0.6

0 5 10 15 20 25 30

Lag

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 28/34
Trend Stationary Processes
↭ A trend stationary process is stationary after removal of its
deterministic trend
↭ Examples: linear or quadratic trends

yt = α + βt + ut
yt = α + β 1 t + β 2 t 2 + ut

with ut a zero mean stationary process

↭ More generally
yt = f ( t ) + u t
E [yt ] = f (t ) ↗= µ constant all t
↭ A trend stationary process is NOT stationary, but

u t = yt → E [ yt ]

is stationary.
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 29/34
Seasonality

↭ Seasonality is di”cult to define in a simple way

but its manifestation is patterns that repeat over years
↭ In macroeconomics:
↭ Retail sales increase prior to Christmas
↭ Industrial production drops in the summer
↭ Some food prices increase in winter

↭ For macroeconometric modelling purposes, often divided into

deterministic & stochastic seasonality

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 30/34
Seasonality
Deterministic Seasonality
↭ Deterministic seasonality is captured through seasonal dummy
variables
↭ Quarterly data example: stationary AR(1) plus 3 seasonal
dummy variables
yt = δ + α1 D1t + α2 D2t + α3 D3t + φyt →1 + ε t , |φ| < 1
Dqt takes value 1 for quarter q & zero otherwise
(arbitrary which seasonal dummy to exclude)
↭ Implication:
E [yt |q = 1 ] : µ1 = δ + α1 + φµ4
E [yt |q = 2 ] : µ2 = δ + α2 + φµ1
E [yt |q = 3 ] : µ3 = δ + α3 + φµ2
E [yt |q = 4 ] : µ4 = δ + φµ3

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 31/34
Seasonality
Stochastic Seasonality I
↭ Stochastic seasonality operates through the dynamics
↭ Quarterly seasonal AR(1) process:

yt = δ + φ4 yt →4 + ε t

Characteristic equation (assuming φ4 > 0)

4 2
! 2
!
z → φ4 = (z → φ4 )(z + φ4 )
! ! 2
!
= (z → φ4 )(z + φ4 )(z + φ4 )
4 4

= 0
⇐
2 real roots, ± 4 φ4
2 ⇐
complex conjugate pair of roots (quadratic z + φ4 = 0)
⇐
all 4 roots have absolute value | 4 φ4 |
↭ Stationary if |φ4 | < 1
Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 32/34
Seasonality
Stochastic Seasonality II
↭ Stationary quarterly SAR(1)
yt = δ + φ4 yt →4 + ε t
↑ seasonality dies out over time &
δ
E [ yt ] =
1 → φ4
↭ Seasonal random walk has φ4 = 1
yt = δ + yt → 4 + ε t
and is a nonstationary process with real roots: ±1
& complex pair of roots with absolute value unity (± i)
↭ Seasonality repeats through nonstationary annual lag
↭ But constant annual change over quarters:
E [ ∆ 4 yt ] = E [ yt → yt → 4 ] = δ

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 33/34
Deterministic vs Stochastic
Deterministic Seasonality Stochasting Seasonality SAR(1)
6

6
4

4
2

y1
y

2
0
−2

0
−4

−2
0 100 200 300 400 0 100 200 300 400

Index Index

Deterministic Seasonality SAR(1) Stochasting Seasonality SAR(1)

1.0

1.0
0.8

0.8
0.6

0.6
0.4
ACF

ACF

0.4
0.2

0.2
0.0

0.0
−0.2

−0.2
−0.4

0 5 10 15 20 25 0 5 10 15 20 25

Lag Lag

Arthur Sinko, University of Manchester ECON60522 Applied Macroeconometrics Univariate Time Series Processes 34/34