ECONOMETRICS II AND QUANTITATIVE TECHNIQUES II

(TIME SERIES)

Dr. Luis Albériko GIL ALAÑA

CONTENT:

1. INTRODUCTION

2. AUTOREGRESSIONS AND MOVING AVERAGE PROCESSES

3. ESTIMATION

4. HYPOTHESIS TESTING

5. MODEL SELECTION

6. SEASONALITY

7. PREDICTION

8. NONSTATIONARY TIME SERIES

9. STRUCTURAL TIME SERIES

10. FREQUENCY DOMAIN AND LONG MEMORY PROCESSES

11. DYNAMIC MODELS

12. MULTIVARIATE TIME SERIES

13. IM`PULSE RESPONSE FUNCTIONS

14. COINTEGRATION

15. ARCH MODELS

16. NONLINEAR TIME SERIES

17. ADDITIONAL TOPICS

Bibliography:

*. G.E.P. Box and G.M. Jenkins, Time series analysis. Forecasting and control. San
Francisco. Holden-Day (1970).

*. Novales, A. Econometría, McGraw-Hill, 2ª Edición (1993).

*. Aznar, A. y Trivez, F.J., Métodos de predicción en Economía II. Análisis de series temporales,
Ariel, Economía (1993).

*. Uriel, E. y A. Peiró, Introducción al análisis de series temporales, Editorial AC (2000).

1
2
1. INTRODUCTION

Definition: TIME SERIES: Collection of observations across time, spaced at equal

intervals of time.

Given a variable, “y”, we denote the time series as

y1, y2, y3, …, yT o alternatively {yt, t = 1, 2, …, T}

Example: U.S. Consumer Price Index 1947m1 – 2004m12:

250

200

150

100

50

0
1947m1 2004m12

Time series análisis is not reduced to Economics. In fact, it is also studied in many other
areas such as Meteorology, Medicine, Biostatistics, Astronomy, Political Sciences, etc.

Objetive of time series:

Find a mathematical model to explain the behaviour of the time series, in order to

*. Predict the future value of the series

*. accept / reject a given theory (based on economics, finance or any other
grounds.).

Basic concepts:

We model the time series as a STOCHASTIC PROCESS. Each observation in a

stochastic process is a random variable and the observations evolve according to some
probabilistic laws.

Definition: Stochastic process: Collection of random variables which are ordered

in time.

3
Time series: y1, y2, y3, …, yT

We treat each observation (data) as if it were a random variable, such that we identify
the time series with the stochastic process

Stochastic process: Y1, Y2, Y3, …, YT

that is, each random variable (Yt), has behind a distribution function, such that

yt → Simple realization of the random variable Yt.

A common assumption in time series is to assume that all variables (that is, all the
observations are i.i.d., = independent and identically distributed).

Given that we treat all the observations as random variables, we need to define their
moments:

*. E(Yt) = E(yt) = μt, t = 1, 2, …, T

*. Var(Yt) = Var(yt) = E(yt – E(yt))2 = E(yt - μt)2, t = 1, 2, …, T

*. Cov(Yt, Yt+k) = Cov(yt, yt+k) = E[(yt – E(yt)) (yt+k – E(yt+k)] =

E[(yt – μt)) (yt+k – μt+k)] = γt, t+k.

However, given that in most cases we only have a single realization available for each
of the random variables, we need to impose some restrictions to make these concepts
feasible. The basic concept here is the one called STATIONARITY.

* Stationarity

A stochastic process (or a time series) is said to be stationary (in a weak sense or
stationary of order 2) if it satisfies the following three properties:

1. E(yt) = μ, for all t.

2. Var(yt) = E((yt – μ)2 = γo for all t.

3. Cov(yt, yt+k) = E[(yt – μ)) (yt+k – μ)] = γk for all t.

4
In other words, the mean and the variance do not depend on time, and the covariance
between any two observations only depends on the distance between them, but not on
the specific location in time.

Thus, given the time series

y1, y2, y3, y4, y5, y6, y7, y8, y9, …, yT

under stationarity,

Cov(y1, y4) = Cov(y2, y5) = Cov(y4, y7) = … = Cov(y10, y13) = etc. = γ3.

To impose stationarity allows us to obtain estimators such as,

1 T
*. ̂ = y =  yt
T t =1

1 T
*. ˆo =  ( yt − y )
2
T t =1

1 T
*. ˆk =  ( yt − y ) ( yt + k − y ).
T t =1

If the stochastic process is stationary and ergodic, these estimators, ̂ , ˆo , y ˆk will be
consistent estimators of μ, γo and γk. (That is, the estimators will tend to be more precise
as the sample size increases).

Though we do not properly define ergodicity, this property requires that observations
which are distant apart will tend to be uncorrelated. That is, a sufficient condition for
ergodicity is:

γk → 0 as k → ∞.

Note that:

Order 2 Stationarity = Covariance Stationarity = Stationarity in Weak Sense

There exists an additional type of stationarity, called “Stationarity in strict sense”. It

imposes that the joint distribution of probabilities in {y1, y2, y3, y4, …, yT} is identical
to the joint distribution of probabilities in {y1+c, y2+c, y3+c, y4+c, …, yT+c} for all T and
c. That is, the distribution function of probabilities is not affected by traslactions in
time:

F(y1, y2, y3, y4, …, yT) = F(y1+c, y2+c, y3+c, y4+c, …, yT+c) for all T and c,

5
where F represents the distribution function of probabilities.

Note that while the definition of weak stationarity is based on moments, the definition
of stationarity in strict sense is based on the distribution function.

Thus, the two definitions are independent and the only relation between them is given
by:

Stationarity in strict sense

+ → Weak stationarity (or of order 2)
Exist the 1 and 2nd order moments
st

Across this course, we mainly work with stationary (in weak sense or order 2) series.
However, most of economic time series are not stationary, and, we will have to
transform them to render stationarity

*. Examples of non-stationary time series:

5,4

5,2

4,8

4,6

4,4

4,2

4
1970m1 2004m12

U.S. Consumer Price Index (monthly data)

1,55

1,5

1,45

1,4

1,35

1,3

1,25

1,2
1 T

Imaginary series (200 observations)

6
 2000

1000

-1000

-2000
7.Jan.86 31.Dec.97

Index of the Hong-Kong (Hang-Seng) stock market index (daily data)

The simplest example of a stationary process is the WHITE NOISE process. Generally,
it is denoted as εt.

*. White Noise process: εt:

E(εt) = 0 for all t.

Var(εt) = σ2 for all t,

Cov(εt, εt+k) = 0 for all t, and for all k ≠ 0.

Examples of White Noise processes

T = 100 observations T = 1000 observations
3 6

2
4

1
2
0
0
-1
-2
-2

-3 -4

-4 -6
1 100 1 1000

Across the course we will focus on stationary time series.

If the time series is nonstationary, we will find a way to transform the time series in
another one that is stationary. Once the new series is found to be stationary, we will

7
look for the mathematical model that best describes the behaviour of the series,
such that the part that is un-modelled is precisely a white noise process..

xt is the original series, (let us supposed that is nonstationary)

yt is the transformed series, supposed to be stationary

Then,

yt = f(*) + εt,

where f is the mathematical function that explain the series and εt is a white noise
process.

*. Example of a stationary stochastic process

yt = εt + θ εt-1, t = 1, 2, … εt ≈ White Noise

It can be proved that (Exercise 1.1):

E(yt) = 0 for all t.

Var(yt) = σ2 (1 + θ2) for all t,

Cov(yt, yt+1) = σ2 θ for all t,

Cov(yt, yt+2) = 0 for all t,

and, in general,

Cov(yt, yt+k) = 0 for all t and all k ≥ 2.

Thus, we observe that the mean, the variance, and the covariance structure do not
depend on time, so the series yt is stationary.

*. Autocovariance function and Autocorrelation function

If yt is stationary,

Cov(yt, yt+k) = γk = γ(k) ≡ autocovariance function

It can be easily proved that (Exercise 1.2):

γk = γ(k) = γ(-k) = γ-k .

8
That is, the autocovariance function is symmetric, so we just focus on values of k ≥ 0.

Similarly, we define

k
k = = autocorrelation function
o

And we can show that (Exercise 1.2)

 k =  ( k ) =  ( −k ) =  − k .

Again, we just focus on values k ≥ 0. You can easily see that o = 1.

In the previous example, (yt = εt + θ εt-1), we get the following values for the
autocorrelation structure (Exercise 1.3):

o = 1,


1 = ,
1 + 2

 k = 0, for all k ≥ 2.

*. (Sample) autocorrelation function

We defined above the following (sample) concepts (based on data):

1 T
̂ = y =  yt (sample mean)
T t =1
1 T
ˆo = 2
 ( yt − y ) (sample variance)
T t =1
1 T
ˆk =  ( yt − y ) ( yt + k − y ). (sample covariance).
T t =1

Similarly, we can define the sample autocorrelation function,

ˆk
ˆ k = , k = 1, 2, ... (1.1)
ˆo

and the graphics that relates ̂ k with k is called CORRELOGRAM.

9
Thus, we have the sample autocorrelation function (Correlogram) → based on data

and the theoretical autocorrelation function → based on a theoretical model

Example:

Suppose we have a theoretical model that is a white noise process (εt). Its
autocorrelation function (theoretical) will be given by the figure in the left-hand-side
below. However, let us suppose now that we have data, which have no structure, and we
compute its correlogram (sample autocorrelation function (1.1)). The values here (right-
hand-side plot), are similar to those given by the white noise model, and therefore we
can conclude that the time series follows a white noise process. (yt = εt)

Theoretical Autocorr. F. (White noise) Correlogram (based on data)

1,2 1,2

1
1
0,8
0,8
0,6
0,6
0,4

0,4 0,2

0,2 0

-0,2
0 0 1 2 3 10
0 1 2 3 10

Autocorrelation function and correlogram for values k = 1, 2, …, 10.

The correlogram tries to reproduce the theoretical autocorrelation function but it does
not imply to be the same. The correlogram will be the instrument employed to identify
the model.

Required notation in the following chapters:

L: Lag-operator: Lyt = yt-1

Lk yt = yt-k

∆: First differences: ∆ = (1 – L)

∆ yt = (1 – L) yt = yt - yt-1

10
2. ARMA PROCESSES

This chapter deals with two types of processes:

1. AutoRegressive Processes (AR)

2. Moving Average Processes (MA)

The combination of the two processes leads to a new process called ARMA
(AutoRegressive + Moving Average).

Before starting with the processes, we define a fundamental Theorem in Time Series
Analysis:

*. Wold Decomposition Theorem

“Any stationary process of order 2, yt, can be uniquely represented as the sum of two
processes mutually uncorrelated, (vt and wt), such that:

yt = vt + wt,

where vt is a deterministic process,

and wt is a stochastic process, such that:

 
 cs  t − s ,
2
wt = with  cs   ,
s =0 s =0

and where εt is a white noise process:

E(εt) = 0 for all t.

Var(εt) = σ2 for all t,

Cov(εt, εt+k) = 0 for all t, and k ≠ 0.”.

In general (and in particular, in this chapter), we focus on the stochastic part (wt).

ARMA PROCESSES

*. Autorregressive processes (AR):

A process yt is said to be autoregressive of order 1 (and denoted by AR(1)) if:

yt =  yt −1 +  t , t = 1, 2, ... (2.1)

11
where εt is a white noise process.

Given (2.1), we can operate recursively such that:

yt −1 =  yt − 2 +  t −1, t = 1, 2, ... (2.2)

and substituting (2.2) in (2.1), we get

yt =  [  yt − 2 +  t −1 ) +  t =  2 yt − 2 +  t +   t −1, (2.3)

Again, using (2.1)

yt − 2 =  yt −3 +  t − 2 , t = 1, 2, ... (2.4)

And substituting (2.4) in (2.3):

yt =  2 [  yt − 3 +  t − 2 ) +  t +   t −1 =  3 yt − 3 +  t +   t −1 +  2  t − 2

and we can go on further such that (after n substitutions)

yt =  n yt − n +  t +   t −1 +  2  t − 2 + ... +  n −1  t − n +1,

or

n −1
yt =  n yt − n +   j  t − j .
j =0

If we impose   1, in (2.1), and make n → ∞, the previous expression becomes,


  t − j ,
j
yt = (2.5)
j =0

since if   1, then  n → 0 as n → ∞.

As a conclusion, we have shown that an autoregressive process of order 1 such as (2.1),

if we impose the condition   1, it can be expressed as (2.5).

An alternative way of getting the same result is using the lag operator:

Starting again with equation (2.1):

12
yt =  yt −1 +  t , t = 1, 2, ...

it can be represented as

yt =  Lyt +  t , t = 1, 2, ...

or

(1 −  L) yt =  t , t = 1, 2, ...

1
yt = t , t = 1, 2, ... (2.6)
(1 −  L)

Next, we use the following mathematical approximation

1
If │x│ < 1,  1 + x + x 2 + x 3 + ...
1− x

such that, if   1,

1
 1 + L +  2 L2 +  3 L3 + ... (2.7)
1 − L

substituting (2.7) in (2.6) we get:

1
yt =  t = (1 + L +  2 L2 +  3 L3 + ...)  t
(1 −  L)

yt =  t +   t −1 +  2 t − 2 +  3 t −3 + ...


  t − j ,
j
yt =
j =0

obtaining the same result as in (2.5).

*. A necessary condition for an AR(1) to be stationary is

  1,

and then (2.1) can be expressed as (2.5).

Next, we examine the statistical properties of the AR(1) processes.

13
Assuming that the AR(1) process is stationary,

yt =  yt −1 +  t ,   1,

we start with the mean,

E ( yt ) = E ( yt −1 +  t ) =  E ( yt −1 ) + E ( t ) .

Noting that in stationary processes, the mean is constant, (μ), and that E(εt) = 0, we get

 =   + 0  (1 −  )  = 0   = E ( yt ) = 0.

Next, we look at the variance,

Var ( yt ) = Var ( yt −1 +  t ) =  2 Var ( yt −1 ) + Var ( t ) + 2  Cov ( yt −1  t ) . (2.8)

Cov ( yt −1  t ) = Cov ( f ( t −1,  t − 2 , ... )  t ) = 0, given that εt is white noise.

again here, given that in stationary processes, the variance of yt is constant (and we
denote it by γo), looking at (2.8) we get:

 o =  2  o +  2 + 0,

2
(1 −  2 )  o =  2   o = Var ( yt ) = .
(1 −  2 )

Now, we look at the autocovariances:

 1 = Cov( yt , yt +1) = Cov( yt ,  yt +  t +1) =  Var ( yt ) + Cov( yt ,  t +1). (2.9)

The last term in (2.9) is 0, given that a stationary AR(1) process depends exclusively on
εt, εt-1, εt-2, … and thus, it is uncorrelated with εt+1. Then, we obtain that:


1 =   o =  2.
1− 2

Similarly, it is easy to prove that (Exercise1.3)

2
 2 =  1 =  2  o =  2.
1− 2

14
 3
3 =   2 = 3  o =  2.
1− 2

For a general k, we have that

k
 k =   k −1 =  2  k − 2 = ... =  2.
1− 2

Once the autocovriance structure is obtained, we can compute the autocorrelation

function:

 o = 1;

1  o
1 = = = ;
o o

2 2 o
2 = = =  2;
o o

and, in general

k k o
k = = = k;
o o

and we see that the decay in the autocorrelation function follows an exponential process.

We show now some graphics of theoretical autocorrelation functions for different

AR(1) processes.

We first suppose that  is a positive value. If that value is close to 1 (  = 0.9) the
decay in the autocorrelation function will be slow. On the contrary, if  is close to 0,
(e.g.  = 0.2), the decay is very fast, and the values become fast close to. On the other
hand, given that  is positive, all the values in the autocorrelation function are positive.

If, on the contrary,  is negative, we get the same exponential decay to 0, depending on
the magnitude of  . However, such decayment oscillates, from postive to negative
values.

15
 Funciones de autocorrelación para distintos procesos AR(1)

 = 0.9  = 0.2
1 0,25
0,9
0,8 0,2
0,7
0,6 0,15
0,5
0,4 0,1
0,3
0,2 0,05
0,1
0 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

 = − 0.9  = − 0.2

1 0,1
0,8
0,05
0,6
0
0,4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
0,2 -0,05

0 -0,1
-0,2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
-0,15
-0,4
-0,2
-0,6
-0,8 -0,25

-1

These previous plots corrspond to the theoretical autocorrelation function. If we are

working with real time series data, we compute the sample autocorrelation function, and
we can get a plot of the correlogram, and, if it is similar to one of the graphics above,
we can argue that the time series follows an AR(1) process.

Later on, we will study higher order autoregressive (AR) processes, e.g., AR(2), AR(3)
and in general AR(p) processes.

*. Moving Average (MA) processes:

A process yt is said to be moving average of order 1 (and denoted by MA(1)) if:

yt =  t +   t −1, t = 1, 2, ... (2.10)

MA processes are ALWAYS stationary.

Again here, we look at its statistical properties.

16
Given (2.10),

E ( yt ) = E (  t +   t −1) = E ( t ) +  E ( t −1) = 0.

Var ( yt ) = Var (  t +   t −1 ) = Var ( t ) +  2 Var ( t −1 ) + 2  Cov( t ,  t −1 ) =

 2 +  2  2 = (1 +  2 )  2 .

We look now at the autocovariance structure:

 1 = Cov( yt , yt +1) = Cov( t +   t −1,  t +1 +   t ) = Cov( t ,  t +1) +  Var ( t ) +

 Cov( t −1,  t +1 ) +  2 Cov( t −1,  t ) =   2 .

 2 = Cov( yt , yt + 2 ) = Cov( t +   t −1,  t + 2 +   t +1) = Cov( t ,  t + 2 ) +  Cov( t ,  t +1) +

 Cov( t −1,  t + 2 ) +  2 Cov( t −1,  t +1 ) = 0.

Similarly, it can be proved that

 3 =  4 =  5 = ... = 0.

In fact, for any value k > 1,

 k = Cov( yt , yt + k ) = Cov( t +   t −1,  t + k +   t + k −1) = Cov( t ,  t + k ) +  Cov( t ,  t + k −1) +

 Cov( t −1,  t + k ) +  2 Cov( t −1,  t + k −1 ) = 0.

Thus, the autocorrelation function of a MA(1) process will be given by:

 o = 1;

1 2 
1 = = = ;
o (1 +  2 )  2 (1 +  2 )

2 0
2 = = = 0;
o o

3 =  4 = 5 = ... = 0.

17
Finally, its graphical representation will consist on a unique value (positive or negative)
depending on the sing of θ.

Autocorrelation functions for different MA(1) processes

 = 0.9  = 0.2

0,6
0,6
0,5 0,5
0,4 0,4

0,3 0,3

0,2
0,2

0,1
0,1
0
0
1 3 5 7 9 11 13 15 17 19
1 3 5 7 9 11 13 15 17 19

 = − 0.9  = − 0.2

0,6 0,6
0,5
0,4
0,4
0,2 0,3
0,2
0
0,1
1 3 5 7 9 11 13 15 17 19
-0,2 0
-0,1 1 3 5 7 9 11 13 15 17 19
-0,4
-0,2
-0,6 -0,3

Once more, if we are working with data, and the correlogram shows a plot similar to
one of these graphics, we can conclude that the time series follows a MA(1) process.

We have said before that the MA(1) processes (and in general any MA process) are
always stationary. Sometimes, it is desirable the process to be invertible, that means that
the process admits an AR(∞) representation. Then, it will require that  < 1
(Invertibility condition). Then, it can be shown that,

yt =  t +   t −1

yt = (1 +  L)  t

1
yt =  t
(1 +  L)

18
 1
yt =  t (2.11)
(1 − (−  ) L)

and given that  < 1:

1
 1 + (− ) L + (− ) 2 L2 + (− )3 L3 + ... = 1 −  L +  2 L2 −  3 L3 + ....
1 − (− ) L

so (2.11) becomnes:

(1 −  L +  2 L2 −  3 L3 + ....) yt =  t

or

yt =  yt −1 −  2 yt − 2 +  3 yt − 3 − .... + t ,

which is an AR(∞) process with a particular structure on the coefficients.

*. AR(2) processes

A process yt is said to be autoregressive of order 2 (and denoted by AR(2)) if:

y t = 1 y t −1 + 2 y t −2 + t , t = 1, 2, ... (2.12)

or, using the lag-operator,

y t = 1 L y t + 2 L2 y t + t , t = 1, 2, ...

(1 − 1 L − 2 L2 ) y t =  t , t = 1, 2, ... (2.13)

1
yt = t , t = 1, 2, ...
(1 − 1 L −  2 L2 )

Next, we look at the conditions that are required for an AR(2) process to be stationary.

A simple way consists of re-writing the polynomial in (2.13) as a function of x instead

of L, and then equalize it to 0, that is,

1 − 1 x −  2 x 2 = 0 .

Next, if we invert it (in the sense that we write which is in the right in the left and
viceversa) and substitute x by z, we get the following equation:

z 2 − 1 z −  2 = 0,

19
which is a second order equation (in standard form), and we can compute its roots, z1
and z2,

− b  b2 − 4 a c 1  12 + 4  2
zi = = ,
2a 2

The stationarity condition is then:

z1  1 y z2  1. (2.14)

In other words, the stationarity condition in AR(2) processes is that “the roots of the
“inverted” polynomial must be both smaller than 1 in absolute value”.

Example:

Determine if the following AR(2) process is stationary or not:

y t = 0.2 y t −1 + y t −2 + t , t = 1, 2, ... (2.15)

Re-writing the model as a function of the lag-operator we get:

(1 − 0.2 L − L2 ) y t =  t , t = 1, 2, ...

and writing the polynomial in terms of x and equalizing it to 0:

1 − 0.2 x − x 2 = 0.

The inverted polynomial as a function of z is:

z 2 − 0.2 z − 1 = 0,

which roots are:

− b  b2 − 4 a c 0.2  0.04 + 4 0.2  2.01

zi = = = = 1.10 and -0.89.
2a 2 2

Given that one of the roots (1.10) is above 1 in absolute value, the process given by
(2.15) is non-stationary.

----------------------------------------------------------------------------------------------------------

20
We have just said that the stationarity condition in AR(2) process is that the roots

− b  b2 − 4 a c 1  12 + 4  2
zi = = ,
2a 2

must be such that

z1  1 and z2  1.

Therefore, we can distinguish three different cases:

a.) if 12 + 4  2  0 , and we obtain two real and distinct roots.

b.) if 12 + 4  2 = 0 , and we obtain two real and identical roots:


( z1 = z 2 = 1 )
2

c.) if 12 + 4  2  0 , and we obtain two complex and distinct roots.

In the latter case, the roots will adopt the form:

z1 = a + b i

z2 = a - b i

and the stationarity conditions states here that:

zi = a2 + b 2  1.

It is easy to prove that:

(1 − 1 L −  2 L2 ) = (1 − z1L) (1 − z 2 L). (2.16)

Using this last relationship (2.16), we get that yt in (2.12) can also be expressed in the
following way:

1
yt = t ,
(1 − 1 L −  2 L2 )

21
  1  1 
y t =      t ,
 1 − z1 L   1 − z 2 L 

and if z1  1 and z2  1 , we obtain that

y t = (1 + z1L + z12 L2 + z13 L3 + ...)(1 + z 2 L + z 22 L2 + z 32 L3 + ...)  t ,

which can be expressed as

y t = (1 + z1L + z 2 L + z12 L2 + z 22 L2 + z1 z 2 L2 + ...)  t ,

y t = (1 + (z1 + z 2 ) L + ( z12 + z 22 + z1 z 2 ) L2 + ...)  t .

Then, yt in (2.12) can be re-written as

y t =  t + 1  t −1 + 2  t −2 + 3  t −3 + ....

where

1 = z1 + z 2 ,

 2 = z12 + z 22 + z1 z 2 ,

3 = z13 + z 32 + z1 z 22 + z 2 z12 , etc.

Example:

yt = − 0.2 yt −1 + 0.08 yt − 2 + t , t = 1, 2, ...

(1 + 0.2 L − 0.08 L2 ) yt = t , t = 1, 2, ...

First we check if this process is stationary or not.

The inverted polynomial is:

z 2 + 0.2 z − 0.08 = 0,

and the roots are,

22
 − b  b2 − 4 a c − 0.2  0.04 + 0.32 − 0.2  0.6
zi = = = = 0.2 and -
2a 2 2
0.4.

Given that the two roots are smaller than 1 in absolute value, the process is stationary.

Furthermore, we know that

(1 + 0.2L − 0.08 L2 ) = (1 − 0.2 L) (1 + 0.4 L) .

Therefore, the process can be re-written as

1 1 1
yt = t = t =
(1 + 0.2 L − 0.08 L2 ) (1 − 0.2 L) (1 + 0.4 L)

yt = (1 + 0.2 L + 0.2 2 L2 + 0.23 L3 + ... ) (1 − 0.4 L + 0.4 2 L2 − 0.43 L3 + ...)  t

such that

yt = (1 + 0.2 L − 0.4 L + 0.04 L2 + 0.16 L2 − 0.08 L2 + ... )  t

yt =  t − 0.2  t −1 + 0.12  t − 2 − ....

Next we look at an alternative way to determine if an AR(2) process is stationary or not.

For this purpose we use now the coefficients of the process. In particular, an AR(2)
process is stationary if the coefficients in the model satisfy the following three
properties:

1. 1 + 2  1

2. 2 − 1  1

3. 2  − 1.

If one of these conditions is not satisfied, the process will not be stationary.

These conditions can be represented graphically,

23
Thus, all the values of 1 and 2 that lie within the triangle will imply stationarity.
Moreover, if we draw, within the triangle, the curve 12 + 4 2 = 0 , we can also
distinguish the real roots from the complex ones.

Finally, we look at the statistical properties of the stationary AR(2) processes:

yt = 1 yt −1 + 2 yt − 2 + t , t = 1, 2, ...

E ( yt ) = E (1 yt −1 + 2 yt − 2 +  t ) = 1 E ( yt −1) + 2 E ( yt − 2 ) + E ( t )

and given that the mean (μ) is supposed to be constant,

24
 = 1 + 2   (1 − 1 − 2 )  = 0  E ( yt ) =  = 0.

Now we look at the variance:

Var( y t ) = Var(1 y t −1 + 2 y t −2 +  t ) = 12 Var( y t −1 ) + 22 Var( y t −2 ) + Var( t )

+ 2 12 Cov( yt −1, yt − 2 ) + 21 Cov( yt −1,  t ) + 22 Cov( yt − 2 ,  t )

Given that in all stationary processes, yt = f ( t ,  t −1, ...), then similarly,

yt −1 = f ( t −1,  t − 2 , ...), and yt − 2 = f ( t − 2 ,  t −3 , ...), so the last two terms in the
previous expression are 0. Moreover, taking into account that Cov(yt-1, yt-2) = γ1, and
given that the variance (γo) is constant, we obtain

 o = 12  o + 22  o + 2 1 2 1 +  2  (1 − 12 − 22 )  o =  2 + 2 1 2 1 

 2 + 2 1  2  1
Var ( y t ) =  o = .
1 − 12 −  22

Next we look at the autocovariance structure:

 1 = Cov( yt , yt −1) = Cov(1 yt −1 + 2 yt − 2 +  t , yt −1) =

1 Var( yt −1) + 2 Cov( yt − 2 , yt −1) + Cov( t , yt −1) = 1  o + 2  1 + Cov( t , yt −1)

and given that Cov( t , yt −1) = 0, we get

1
 1 = 1  o + 2  1  (1 − 2 )  1 = 1  o  1 =  o.
1 − 2

 2 = Cov( yt , yt − 2 ) = Cov(1 yt −1 + 2 yt − 2 +  t , yt − 2 ) =

1 Cov( yt −1, yt − 2 ) + 2 Var ( yt − 2 ) + Cov( t , yt − 2 ) = 1  1 + 2  0 + Cov( t , yt − 2 )

and given that Cov( t , yt − 2 ) = 0, we obtain

 1   12  12 + 2 − 22

 2 = 1  1 + 2  o = 1   o + 2  o =  
+ 2  o = o .
 1 − 2   1 − 2  1 − 2
 

Similarly we can calculate γ3,

 3 = 1  2 + 2  1,

25
and, in general,

 s = 1  s −1 + 2  s − 2 , s = 0,1, ... (2.17)

Dividing (2.17) by γo, we obtain the structure for the autocorrelations,

s  
= 1 s −1 + 2 s − 2 , s = 0, 1, ...
o o o

 s = 1  s −1 + 2  s − 2 , s = 0,1, ... (2.18)

such that,

o = 1.

1
1 = .
1 − 2

12
2 = + 2 , etc.
1 − 2

These values can be obtained from (2.18), taking into account that ρk = ρ-k and ρo = 1.

The graphical representation of the autocorrelation function in AR(2) processes is very

sensitive to the characterization of the coefficients of the process. Let’s see some
examples:

Autocorrelation functions for some AR(2) processes

1 = 0.5;  2 = 0.3 1 = 0.3;  2 = − 0.5

1,2 1,2

1
0,8

0,8
0,4
0,6
0
0,4

-0,4
0,2

0 -0,8
0 10 20 0 10 20

26
*. MA(2) processes

A process yt is said to be moving average of order 2 (and denoted by MA(2)) if:

yt =  t + 1  t −1 +  2  t − 2 , t = 1, 2, ... (2.19)

or

yt =  t + 1 L  t +  2 L2  t , t = 1, 2, ...

yt = (1 + 1 L +  2 L2 )  t , t = 1, 2, ...

The MA processes are always stationary.

It can be easily proved that

E ( yt ) =  = 0.

Var ( yt ) =  o = (1 + 12 +  22 )  2 .

Cov ( yt , yt +1 ) =  1 = (1 + 1  2 )  2 .

Cov( yt , yt + 2 ) =  2 =  2  2 .

Cov( yt , yt + s ) =  s = 0, for all s > 2.

Therefore, the autocorrelation function is given by:

o = 1.

1 + 1  2
1 = .
1 + 12 +  22

2
2 = .
1 + 12 +  22

 s = 0, for all s > 2.

Thus, the autocorrelation function of MA(2) processes is null for lags higher than 2.
Let’s see some examples in graphical terms:

27
 Autocorrelation functions of MA(2) processes

1 = 0.5;  2 = 0.2 1 = − 0.6;  2 = 0.3

1,2 1,2

1 0,8

0,8
0,4
0,6
0
0,4
-0,4
0,2

0 -0,8
0 1 2

*. ARMA(1, 1) processes

A process yt is ARMA(1, 1) if:

yt =  yt −1 +  t +   t −1, t = 1, 2, ... (2.20)

or

yt =  L yt + t +  L t , t = 1, 2, ...

(1 −  L) yt = (1 +  L)  t , t = 1, 2, ...

The ARMA(1, 1) processes are:

stationary if:   1,

and invertible if:   1.

Next we look at the statistical properties of the stationary ARMA(1, 1) processes.

Given (2.20):

E ( yt ) = E ( yt −1 +  t +   t −1 ) =  E ( yt −1 ) + E ( t ) +  E ( t −1 ),

 =  + 0 + 0  (1 −  )  = 0  E ( yt ) =  = 0.

28
Var ( yt ) = Var ( yt −1 +  t +   t −1 ) =  2 Var ( yt −1 ) + Var ( t ) +  2 Var ( t −1 ) +

2 Cov( yt −1,  t ) + 2 Cov( t ,  t −1 ) + 2  Cov( yt −1,  t −1 ).

Again here, Cov( yt −1,  t ) = 0, and Cov( t ,  t −1 ) = 0. We focus first on the last term,

Cov( y t −1 ,  t −1 ) = Cov( y t −2 +  t −1 +   t −2 ,  t −1 ) =  2 .

We get:

(1 +  2 + 2   ) 2
 o =  2  o +  2 +  2 2 + 2    2   o = Var ( yt ) = 
1 − 2

 1 = Cov( yt , yt +1) = Cov( yt ,  yt +  t +1 +   t ) =  Var( yt ) + Cov( yt ,  t +1) +

 Cov( yt ,  t ).

Taking into account that Cov( yt ,  t +1 ) = 0, and

Cov( yt ,  t ) = Cov( yt −1 +  t +   t −1,  t ) =  2 , we get that:

1 =   o +   2.

 2 = Cov( yt , yt + 2 ) = Cov( yt ,  yt +1 +  t + 2 +   t +1) =  Cov( yt , yt +1) + Cov( yt ,  t + 2 ) +

 Cov( yt ,  t +1) =   1.

Similarly,

 3 = Cov( yt , yt + 3 ) =   2 = 3  o + 2  2 ,

and, in general, for all s > 1, we have that

 s = Cov( yt , yt + s ) =   s −1 =  s  o +  s −1   2 .

The autocorrelation structure is then given by:

 o = 1;

29
 1  o +   2 2
1 = = =  + ;
o o o

2  1
2 = = =  1;
o o

and, in general, for all s > 1

s
s = =   s −1.
o
Thus, for s > 1, we get the same structure as in purely AR(1) processes. Generalizing all
the previous processes, we have

*. AR(p) processes

A process yt is autoregressive of order p (and denoted by AR(p)) if:

yt = 1 yt −1 + 2 yt − 2 + ... +  p yt − p +  t , t = 1, 2, ... (2.21)

or

yt = 1 L yt + 2 L2 yt + ... +  p L p yt +  t , t = 1, 2, ...

(1 − 1 L − 2 L2 − ... −  p L p ) yt = t , t = 1, 2, ...

 p ( L) yt =  t , t = 1, 2, ...

where  p (L) = (1 − 1 L − 2 L2 − ... −  p Lp ).

An AR(p) process is stationary if the roots of the polynomial:

( z p − 1 z p −1 − 2 z p − 2 − ... −  p )

are smaller than 1 in absolute value. That is,

z1  1; z2  1; .... zp  1.

Next we look at the autocovariance and the autocorrelation structures. Given (2.21):

30
E ( yt ) = 1 E ( yt −1 ) + 2 E ( yt − 2 ) + ... +  p E ( yt − p ) + E ( t )

 = 1  + 2  + ... +  p  + 0  (1 − 1 − 2 .... −  p )  = 0   = 0.

Instead of looking at the variance, (γo), we look at the autocovariances for a given value
k. For this purpose, we first multiply both sides in (2.21) by yt-k, such that

yt yt − k = 1 yt −1 yt − k + 2 yt − 2 yt − k + ... +  p yt − p yt − k +  t yt − k ,

and next we calculate the mathematical expectation, (E(.)), in both sides of the previous
equality, that is,

E ( yt yt − k ) = 1 E ( yt −1 yt − k ) + 2 E ( yt − 2 yt − k ) + ... +  p E ( yt − p yt − k ) + E (  t yt − k ).

Taking into account that the means are 0, we can re-write that expression as:

Cov ( yt yt − k ) = 1 Cov ( yt −1 yt − k ) + 2 Cov ( yt − 2 yt − k ) + ... +  p Cov ( yt − p yt − k ) + Cov (  t yt − k ),

and given that the last term is 0, we get

 k = 1  k −1 + 2  k − 2 + ... +  p  k − p . (2.22)

Dividing (2.22) by γo,

 k = 1  k −1 + 2  k − 2 + ... +  p  k − p ,

and calculating the values in the last expression for k = 1, 2, 3, …p:

1 = 1 + 2 1 + ... +  p  p −1

 2 = 1 1 + 2 + ... +  p  p − 2

 3 = 1  2 + 2 1 + ... +  p  p − 3

...

 p = 1  p −1 + 2  p − 2 + ... +  p .

That is, we have p equations to solve p unknowns (ρ1, …, ρp), which is sufficient to
calculate the autocorrelation function. This system of equations is known as the YULE-
WALKER equations.

31
*. MA(q) processes

A process yt is moving average of order q (and denoted by MA(q)) if:

yt =  t + 1  t −1 +  2  t − 2 + ... +  q t − q , t = 1, 2, ...

or

yt =  t + 1 L  t +  2 L2  t + ... +  q Lq t , t = 1, 2, ...

yt = (1 + 1 L +  2 L2 + .... +  q Lq )  t , t = 1, 2, ...

yt =  q ( L)  t , t = 1, 2, ....

where  q (L) = (1 + 1 L +  2 L2 + ... +  q Lq ).

Remember once more that all MA processes are always statonary.

.

A MA(q) process is invertible if the roots of the polynomial:

( z q + 1 z q −1 +  2 z q − 2 + ... +  q )

are smaller than 1 in absolute value. That is,

z1  1; z2  1; .... zq  1.

The autocovariance structure is given by:

 k = Cov( yt yt + k ) = Cov ( t + 1  t −1 +  2  t − 2 + ... +  q  t − q ,  t + k + 1  t + k −1 + ... +  q  t + k − q )

and it can easily be proved that:

k =
 2 (1 + 12 +  22 + ... +  q2 ) if k = 0,
 2 ( k + 1  k +1 +  2 k + 2 + ... +  q k + q ) if k = 1, 2, …, q,
0 if k > q.

32
Thus, the autocorrelation function is given by:

k =

1 if k = 0,
( k + 1  k +1 +  2 k + 2 + ... +  q  k + q )
if k = 1, 2, …, q,
(1 + 12 +  22 + ... +  q2 )
0 if k > q.

Thus, the autocorrelation function of a MA(q) process is null for lags above q. This is a
very useful results noting that when we compute the correlogram of a given time series,
if it is close to 0 from a given lag, we can easily identify it as a MA process. For
instance, if the correlogram adopts the graphic in the left (Series 1) we can suggest that
the model that is behind the data is a MA(5), while the correlogram in the right hand
side might correspond to a MA(2) process.

Correlograms corresponding to imaginery time series

Series 1 Series 2

1,2 1,2

0,8
0,8
0,4

3
0 0,4
0 1 2 4 5 0 1 2

-0,4 4
0
-0,8 0 1 2

-1,2 -0,4

Finally, we look at the most general case:

*. ARMA(p, q) process

A process yt is ARMA(p, q) if:

yt = 1 yt −1 + ... +  p yt − p +  t + 1  t −1 + ... +  q t − q , t = 1, 2, ...

or

33
yt = 1 L yt + .... +  p L p yt +  t + 1 L  t + ... +  q Lq t , t = 1, 2, ...

(1 − 1L − ... −  p L p ) yt = (1 + 1 L + ... +  q Lq )  t , t = 1, 2, ...

 p ( L) yt =  q ( L)  t , t = 1, 2, ....

An ARMA(p, q) process is stationary if the p roots of the polynomial:

( z p − 1 z p −1 − 2 z p − 2 − ... −  p )

are smaller than 1 in absolute value. Similarly, the process will be invertible if the q
roots of the polynomial:

( z q + 1 z q −1 +  2 z q − 2 + ... +  q )

are smaller than 1 in absolute value.

Given the stationary ARMA(p, q) process, it can be proved that:

 o = 1  1 + 2  2 + ... +  p p +  2 (1 + 12 +  22 + ... +  q2 )

and  k = Cov( yt yt − k ) =

Cov (1 yt −1 + 2 yt − 2 + .... +  p yt − p +  t + 1  t −1 +  2  t − 2 + ... +  q  t − q , yt − k )

such that  k =

1  k −1 + 2  k − 2 + ... +  p k − p + Cov( t , yt − k ) + 1 Cov( t −1, yt − k ) + ... +  q Cov( t − q , yt − k )

Then, for values k > q, we obtain

 k = 1  k −1 + 2  k − 2 + ... +  p  k − p k = q + 1, q + 2, …

That is, for lags above q, the autocorrelation function does not take into account the
contribution from the MA part, and thus, its behaviour from lag q is the same as in an
AR(p) process.

34
*. Additional concepts in Chapter 2

Appendix 1: Partial correlogram

We have seen before that it is quite easy to detect MA(q) processes from the
correlogram. In fact, it is simply to observe from which lag the values are close to 0.
Thus, for example, if the values in the correlogram are approximately 0 in lag 4 and in
all the following ones, the time series may be described as a MA(3) process.

However, to detect AR(p) processes from the correlogram is not an easy task since the
autocorrelation function substantially changes depending on the parameters of the
model Here, we can use another instrument that is called partial correlogram.

We first consider the following processes:

yt = 11 yt −1 +  t ,

yt = 21 yt −1 + 22 yt − 2 +  t ,

yt = 31 yt −1 + 32 yt − 2 + 33 yt −3 +  t ,

and sucesively up to

yt =  p1 yt −1 +  p 2 yt − 2 + ... +  pp yt − p +  t .

As you can see, all of them are AR processes with orders that are increasing from one
process to another. The sucession of values

11, 22 , 33 , ... ,  pp

is called the Partial Autocorrelation Function, and is formed by the series of the last AR
coefficients which are sucessively higher.

If, instead of ss , we use the estimators of such values, ˆss , the sucession of values

ˆ11, ˆ22 , ˆ33 , ... , ˆpp

is called the Partial Correlogram.

The partial correlogram is a very useful instrument to detect purely AR processes,

noting that if

ˆss  0 for all s > p

35
the time series will be an AR(p) process.

Example:

Let us suppose that the partial correlograms of the following series 3 and 4 adopt the
following form
:

Partial correlograms corresponding to imaginery series

Series 3 Series 4

1,2 1,2

0,8
0,8
0,4

3
0 0,4
0 1 2 4 5 0 1 2

-0,4 4
0
-0,8 0 1 2

-1,2 -0,4

Then, we can conclude by saying that Series 3 follows an AR(5) process, while Series 4
follows an AR(2) process.

Appendix 2: Matritial representation

Given the time series {y1, y2, y3, …, yT}, we can study its statistical properties from a
matritial viewpoint.

In general, we call “y” the (Tx1) vector of the observations,

36
  y1 
 
y 
y =  2
−
 
y 
 T

and, in general, we have that

y  N (m,  )
(Tx1) (Tx1) (TxT )

where

 E ( y1 )   1 
   
 E ( y2 )   2 
m =  =
−  − 
   
 E( y )   
 T   T

 V ( y1 ) Cov ( y1 y2 ) ... Cov ( y1 yT )   11 12 ... 1T 

   
 Cov ( y2 y1 ) V ( y2 ) ... Cov ( y2 yt )    21  22 ...  2T 
 =   =  ... .
... ... ... ... ... ... ... 
   
 Cov ( y y ) Cov ( y y ) V ( yT )   T 2 ... TT 
 T 1 T 2 ...  T1

Now, if we assume stationarity of order 2(i.e., covariance stationary), that is,

*. E(yt) = μ, for all t.

*. Var(yt) = E((yt – μ)2 = γo for all t.

*. Cov(yt, yt+k) = E[(yt – μ)) (yt+k – μ)] = γk for all t.

the previous expressions become:

 E ( y1 )  
   
 E ( y2 )  
m = 
−  = −  = 
   
 E( y )  
 T   

37
  V ( y1 ) Cov ( y1 y2 ) ... Cov ( y1 yT )   o 1  2 ...  T −1 
   
 Cov ( y2 y1 ) V ( y2 ) ... Cov ( y2 yt )  1 o  1 ...  T − 2 
 =   =  ... .
... ... ... ... ... ... ... ... 
   
 Cov ( y y ) Cov ( y y )     
 T 1 T 2 ... V ( yT )   T −1 T − 2 ... ... o 

This type of matrices, (where all the elements across the diagonals are equal) are called
TOEPLITZ matrices.

Adic. 3: Nonstationarity

At the beginning of this course we said that most economic time series are of a
nonstationary nature.

Here we briefly present two alterantive ways of transforming a time series that is
nonstationary into stationary.

A usual way (at least until the 80s) of describing the nonstationarity was to employ
deterministic functions (generally linear or quadratic) in time. That is, you can regress yt
on a variable “t” that describes a linear trend, or to regress it on the variables “t” y “t2”
(quadratic function), assuming that the resulting residuals are stationary. Thus, for
example, if the time series is of form

5,4

5,2

4,8

4,6

4,4

4,2

4
1970m1 2004m12

a way of modelling this series would be:

yt =  + t + ut

such that once the parameters are estimated, ( ̂ y ̂ ) , the resulting residuals are
stationary and can be modelled using the ARMA models:

 p ( L ) ut =  q ( L )  t . (2.23)

38
In other words, what we do is to use a linear trend such that the discrepancies between
the observed values and the estimated ones form the linear trend are stationary. This
type of transformations is usually called the “deterministic approach”.

An alternative way of transforming a time series into statitionarity is to take first

difeferences. That is, instead of working directly with the original data, yt, we do it with
their first differecnes, yt - yt-1, or

ut = (1 - L) yt = yt - yt-1 .

Assuming that ut is now stationary, we proceed to study its ARMA structure in the same
way as in (2.23). This approach is called the “stochastic approach” and will be studied
in detail in future chapters.

Giving some ideas in advance, we say that a process yt is ARIMA(p, d, q) if:

 p ( L) (1 − L) d yt =  q ( L)  t .

where
 p (L) is the AR polynomial of order p,

 q (L) is the MA polynomial of order q,

and d is the number of differences required to get stationarity (usually d

= 1 and/or sometimes 2).

39
40