ECON7350

Univariate Time Series - II

Eric Eisenstat

The University of Queensland

Lecture 3

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 1 / 24
Estimation of Univariate Time Series Models

Recommended readings

Author Title Chapter Call No

Enders Applied Econometric 2 HB139 .E55 2015
Time Series, 4e
Verbeek A Guide to Modern 8.7, 8.8 HB139 .V465 2012
Econometrics

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 2 / 24
Univariate Autoregressive Moving Average Models

ARMA(1, 0)
yt = a0 + a1 yt−1 + εt

ARMA(0, 1)
yt = a0 + εt + β1 εt−1

ARMA(1, 1)
yt = a0 + a1 yt−1 + εt + β1 εt−1

ARMA(2, 2)

yt = a0 + a1 yt−1 + a2 yt−2 + εt + β1 εt−1 + β2 εt−2

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 3 / 24
ACF and PACF

The ACF and PACF can be helpful starting point in identifying reasonable
model structures.

MODEL ACF PACF

ARMA(1, 0) Decays to zero One non-zero peak
ARMA(0, 1) One non-zero peak Decays to zero
ARMA(1, 1) Decays to zero Decays to zero
ARMA(2, 0) Decays to zero Two non-zero peaks

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 4 / 24
Coles Myers Dividend Yield Series, 1983Q3–2003Q3

DIVYIELD
8

1
84 86 88 90 92 94 96 98 00 02

Does the plot of the series show any apparent trend?

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 5 / 24
Model Identification

Date: 08/08/16 Time: 14:07

Sample: 1983Q3 2003Q3
Included observations: 81

Autocorrelation Partial Correlation AC PAC Q-Stat Prob

1 0.818 0.818 56.284 0.000

2 0.662 -0.023 93.580 0.000
3 0.548 0.038 119.44 0.000
4 0.433 -0.062 135.85 0.000
5 0.345 0.010 146.35 0.000
6 0.195 -0.245 149.76 0.000
7 0.070 -0.043 150.21 0.000
8 -0.029 -0.068 150.29 0.000
9 -0.065 0.126 150.68 0.000
10 -0.095 -0.046 151.54 0.000
11 -0.204 -0.237 155.55 0.000
12 -0.280 -0.056 163.21 0.000
13 -0.324 -0.029 173.57 0.000
14 -0.360 -0.103 186.59 0.000
15 -0.323 0.155 197.20 0.000
16 -0.297 0.032 206.31 0.000
17 -0.246 0.100 212.64 0.000
18 -0.210 -0.108 217.33 0.000
19 -0.155 0.017 219.93 0.000
20 -0.088 -0.031 220.77 0.000
21 -0.023 0.124 220.83 0.000
22 0.040 -0.018 221.02 0.000
23 0.028 -0.131 221.11 0.000
24 0.001 -0.114 221.11 0.000

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 6 / 24
Model Identification for the Coles Myers Dividend Yield
Data

The SPACF appears to have a single non-zero peak.

The SACF appears to be decaying.

Is there a particularly suitable model suggested?

Good practice: even if a “most suitable” model stands out, continue

working with a range of models around it!

Further steps in the process of model selection require estimation.

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 7 / 24
Estimation and Diagnostics
Recall that a pure AR model (i.e. with no MA lags, or equivalently
ARMA(p, 0)) can be estimated using OLS.

Including MA lags requires either Non-linear Least Squares (NLLS) or

Maximum Likelihood (MLE).

NLLS chooses parameters α0 , . . . , αp , β1 , . . . , βq , σε2 to minimize the

non-linear sum of squares.

MLE chooses parameters α0 , . . . , αp , β1 , . . . , βq , σε2 to maximize the

likelihood.
A likelihood is the joint density of the observed data (e.g. y1 , . . . , yT )
as a function of the model parameters.
Assuming a distribution for the errors implies a likelihood for data.
See Appendix 2.1 of Enders for the form of the likelihood (and
log-likelihood) of the ARMA(p, q).

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 8 / 24
Estimation and Diagnostics

AR(1) MA(1)
Dependent Variable: DIVYIELD Dependent Variable: DIVYIELD
Method: ARMA Maximum Likelihood (BFGS) Method: ARMA Maximum Likelihood (BFGS)
Date: 03/10/16 Time: 15:42 Date: 03/10/16 Time: 15:40
Sample: 1983Q3 2003Q3 Sample: 1983Q3 2003Q3
Included observations: 81 Included observations: 81
Convergence achieved after 5 iterations Convergence achieved after 7 iterations
Coefficient covariance computed using outer product of gradients Coefficient covariance computed using outer product of gradients

Variable Coefficient Std. Error t-Statistic Prob. Variable Coefficient Std. Error t-Statistic Prob.

C 3.916264 0.445945 8.781954 0.0000 C 3.932779 0.166361 23.63996 0.0000

AR(1) 0.850219 0.072178 11.77952 0.0000 MA(1) 0.790423 0.077826 10.15624 0.0000
SIGMASQ 0.434378 0.051893 8.370618 0.0000 SIGMASQ 0.690675 0.102480 6.739628 0.0000

R-squared 0.705297 Mean dependent var 3.927654 R-squared 0.531414 Mean dependent var 3.927654
Adjusted R-squared 0.697741 S.D. dependent var 1.221629 Adjusted R-squared 0.519398 S.D. dependent var 1.221629
S.E. of regression 0.671628 Akaike info criterion 2.093954 S.E. of regression 0.846900 Akaike info criterion 2.553966
Sum squared resid 35.18461 Schwarz criterion 2.182637 Sum squared resid 55.94466 Schwarz criterion 2.642650
Log likelihood -81.80513 Hannan-Quinn criter. 2.129535 Log likelihood -100.4356 Hannan-Quinn criter. 2.589547
F-statistic 93.33682 Durbin-Watson stat 1.917667 F-statistic 44.22903 Durbin-Watson stat 1.324738
Prob(F-statistic) 0.000000 Prob(F-statistic) 0.000000

Inverted AR Roots .85 Inverted MA Roots -.79

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 9 / 24
Criteria for Model Selection
Once a set of models is estimated, we can use information criteria to
choose amongst the models in a more formal way and reduce the set.

Two commonly used information criteria are: Akaike’s Information

Criterion (AIC) and Bayesian Information Criterion (BIC, or sometimes SC,
SBC, SBIC).

All Information criteria quantify a penalty for lack of fit and / or

over-parameterizations.
When comparing models using information criteria, lower is better.
The trade off is fit versus parsimony.

Both AIC and BIC are likelihood based:

p+q+1 p+q+1
bε2 + 2
AIC = ln σ bε2 + ln T
BIC = ln σ .
T T
They both sum the log of the estimated residual variance (i.e. measure of
fit) and estimate of the number of free parameters.
Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 10 / 24
Model Identification for the Coles Myers Dividend Yield
Data

The AIC is 2.554 for MA(1) and 2.094 for AR(1). Which is better?

The BIC is 2.642 for MA(1) and 2.182 for AR(1). Which is better?

Do the AIC and BIC agree on the preferred specification? How does this
compare to what was suggested by the SACF and SPACF?

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 11 / 24
Analyzing the Residuals

The residual εt in a ARMA(p, q) model is theoretically uncorrelated. We

can test for this given an estimated ARMA model!

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 12 / 24
The Q-Statistic
A process with zero autocorrelation is called white noise.

Test if a process is white noise using the Ljung-Box (1978) Q-statistic.

joint test that the first K autocorrelations (ρk ) are not significant;
H0 : ρ1 = ρ2 = · · · = ρK = 0 versus H1 : ?

Test statistic is computed as

K
X rk2
QK = T (T + 2) ,
T −k
k=1

where rk is the sample autocorrelation at lag k, and K is the number of

autocorrelation lags being tested.

Sampling distribution of QK :
observed time series: QK ∼ χ2K ;
estimated residuals of an ARMA(p, q): QK ∼ χ2K−p−q .
Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 13 / 24
Inference With Time-Series Data

What can we do with an estimated ARMA(p, q) for a sample y1 , . . . , yT ?

Forecast: predict future values yT +1 , . . . , yT +h .

Impulse Response Functions: analyze response of variables to

unforeseeable shocks.

More possibilities for multivariate dynamic systems.

The end goal of model identification: obtain “accurate” forecasts and / or

impulse response functions.
set of models too wide ⇒ more uncertainty and less accuracy;
set of models too narrow ⇒ more bias and less accuracy;
we want to use formal tools strike an “optimal balance”, but it is not
a perfect science!

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 14 / 24
Strategy to Fitting ARMA(p, q)

Identification
1. Plot the data and decide an appropriate process to begin with.
2. Use the SACF and SPACF to suggest how many AR or MA terms.
3. Construct a set of adequate ARMA models.
Estimation and Diagnostics
4. Estimate each model in the constructed set.
5. Reduce this set by choosing models with “low” IC metrics (AIC, BIC).
6. Look at the SACF, SPACF and Q-statistic for the residuals; eliminate
any model for which a white noise residual is rejected.
7. If the resulting set is “too small”, add models with more AR and MA
lags, then repeat Steps 4-6.
Inference
8. Compare results across the final set of models; interpret accordingly.

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 15 / 24
Lag Operator Notation

To discuss forecasting and impulse response function, it helps to define

some notation.

Definition
The lag operator L applied to a stochastic process {yt } transforms a
realisation at time t into a realisation at time t − 1, i.e.

yt−1 = Lyt .

This helps us write polynomials in the lag operator:

a(L) = 1 − a1 L − · · · − ap Lp ,
β(L) = 1 + β1 L + · · · + βq Lq .

Then, the ARMA(p, q) can be concisely expressed a(L)yt = a0 + β(L)εt .

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 16 / 24
AR and MA representations of the ARMA(p, q)
If the ARMA(p, q) is stable, then a(L) can be inverted to produce the pure
MA representation yt = a0 /a(1) + θ(L)εt , where

β(L)
a(1) = 1 − a1 − · · · − ap , θ(L) = ,
a(L)

If the ARMA(p, q) is invertible, then β(L) can be inverted to produce the

pure AR representation φ(L)yt = a0 /b(1) + εt , where

a(L)
b(1) = 1 + b1 + · · · + bq , φ(L) = .
β(L)

If the ARMA(p, q) is stable and invertible, then we will also refer to

yt = a0 /a(1) + θ(L)εt as the Wold representation.
Every stationary time series has a Wold representation!
Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 17 / 24
Forecasting
Recall that given observations y1 , . . . , yT , the optimal predictor of a future
value yT +h is the conditional expectation ybT +h ≡ E(yT +h | y1 , . . . , yT ).

Given parameter estimates of an invertible ARMA(p, q), we can estimate

residuals {b
εt } using εbt = φ(L)y
b t−b
a0 /bb(1) and yb0 = · · · = yb−∞ = 0.

Then, generate point forecasts recursively:

ybT +1 = a0 + a1 yT + · · · + ap yT −p+1 + β1 εbT + · · · + βq εbT −q+1 ,

ybT +2 = a0 + a1 ybT +1 + · · · + ap yT −p+2 + β2 εbT + · · · + βq εbT −q+2 ,
..
.
ybT +q+1 = a0 + a1 ybT +q + · · · + ap yT −p+q+1 , if p > q
..
.
ybT +h = a0 + a1 ybT +h−1 + · · · + ap ybT −p+h , h > max{p, q}.

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 18 / 24
Forecast Uncertainty
But point forecasts on their own are difficult to interpret.

To make forecasts useful, we need to quantify uncertainty. Two sources of

uncertainty:
Uncertainty due to unknown future, i.e. future errors εT +1 , . . . , εT +h .
Uncertainty due to parameter estimation, i.e. using
a0 , . . . , b
b ap , βb1 , . . . , βbq instead of the “true” a0 , . . . , ap , β1 , . . . , βq .

If all parameters were known, the forecast error variance would be:

Var(yT +h − ybT +h ) = σε2 (1 + θ12 + · · · + θh−1

2
).

1−a2h
Example: For an AR(1), Var(yT +h − ybT +h ) = σε2 1
1−a21
.

Question: what happens as h −→ ∞?

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 19 / 24
Impulse Response Functions

Policy is often concerned with how a variable responds to a shock.

If we have a positive, unanticipated shock to GDP, how will GDP
respond in the future?
We consider the impulse response path: the difference between the
expected path with the shock and the expected path without the
shock.

∂yt+h
The impulse response function (IRF) is the partial derivative ∂εt for all
h ≥ 0. In a linear model, this is equivalent to

E(yt+h | εt = 1, εt−1 , εt−2 , . . . )

− E(yt+h | εt = 0, εt−1 , εt−2 , . . . ) = θh .

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 20 / 24
Estimating Impulse Response Functions
Given an estimated ARMA(p, q), the IRF can be obtained by either
iteratively computing ybt+h for εt = 1 and iteratively computing ȳt+h
for εt = 0, then setting IR(h) = ybt+h − ȳt+h , or
by computing θ(L) = β(L)/a(L).

Note that for IRFs deterministic terms (such as a0 ) do not matter—we

can ignore them.

θ(L) can be derived from a(L) and β(L) using the method of
undetermined coefficients (see Enders pp. 33–42).

If the ARMA(p, q) is stable, then IRFs decay to zero; otherwise they either
explode to ±∞ or, in a special case that will be discussed soon, converge
to a non-zero constant.

Uncertainty in IRFs is only due to uncertainty in parameter estimation.

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 21 / 24
Example: ARMA(2, 1)

Suppose we estimate the model

yt = a1 yt−1 + a2 yt−2 + εt + β1 εt−1 ,

yt = 1.2yt−1 − 0.7yt−2 + εt + 0.3εt−1 .

In this case, a(L) = 1 − 1.2L + 0.7L2 and β(L) = 1 + 0.3L.

We wish to find

θ(L) = (1 + 0.3L)/(1 − 1.2L + 0.7L2 ) = 1 + θ1 L + θ2 L2 + · · · .

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 22 / 24
Method of Undetermined Coefficients for the ARMA(2, 1)

(1 + 0.3L)/(1 − 1.2L + 0.7L2 ) = 1 + θ1 L + θ2 L2 + · · · ,

1 + 0.3L = (1 − 1.2L + 0.7L2 )(1 + θ1 L + θ2 L2 + · · · ),
1 + 0.3L = 1 + θ1 L + θ2 L2 + · · ·
− 1.2L − 1.2θ1 L2 − 1.2θ2 L3 − · · ·
+ 0.7L2 + 0.7θ1 L3 + 0.7θ2 L4 − · · · ,

which leads to

L0 : 1 = 1,
L1 : 0.3 = θ1 − 1.2 ⇒ θ1 = 1.5,
2
L : 0 = θ2 − 1.2θ1 + 0.7 ⇒ θ2 = 1.1,
3
L : 0 = θ3 − 1.2θ2 + 0.7θ1 ⇒ θ3 = 0.27,
k
L : 0 = θk − 1.2θk−1 + 0.7θk−2 ⇒ θk = 1.2θk−1 − 0.7θk−2 .

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 23 / 24
IRF to a shock εt = 1

1.5

0.5

−0.5

−1
0 10 20 30 40

Eric Eisenstat (School of Economics) Applied Econometrics for Macro and Finance Week 3 24 / 24