XÍV CONTENTS

Appendix 6.1: Inference on a Cointegrating Vector 378

Appendix 6.2: Characteristic Roots, Stability, and Rank 381

Chapter 7
1
2
3
4
NONLINEAR TIME-SERIES MODELS
Linear versus Nonlinear Adjustment
Simple Extensions of the ARMA Model
Threshold Autoregressive Models 393
Extensions and Other Nonlinear Models
387
39O

399
387

CHAPTER 1
5 Testing for Nonlinearity 406
6 Estimates of Regime Switching Models 414 DIFFERENCE EQUATIONS
7 Generalized Impulse Responses and Forecasting 423
8 Unit Roots and Nonlinearity 429
9 Surnmary and Conclusions 434
Questions and Exercises 435
Endnotes 438 INTRODUCTIOiM __________________________________ _ _
The theory of difference equations underhés all of the time-series methods employed in
STATISTICAL TABLES 439 later chapters of this text. It is fair to say that time-series econometrics is concerned with
the estimation of difference equations containing stochastic components. The tradi- -
REFERENCES 445 INDEX tional use of time-series analysis was to forecast the time path of a variable. Uncóvering
the dynamic path of a series improves forecasts since the predictable components of the
452 series can be extrapolated into the future. The growing interest in economic dynamics
has given a new emphasis to time-series econometrics. Stochastic difference equations
arise quite naturaliy from dynamic economic models. Appropriately estimated equa-
tions can be used for the interpretad on of economic data and for hypothesis testing. This
introductory chapter has three aims:
1. Explain how stochastic difference^equations can be used for forecasting and
illustrate how such equations can*aríse from familiar economic models. The
chapter is not meant to be a treatise on the theory of difference equations.
Only those techniques that are essential to the appropriate estimation of lin
ear time-series models are presented. This chapter focúses on single equa-
tion models; multivariate models are considered in Chapters 5 and 6.
2. Explain what it means to solve a difference equation. The soiution wiil deter
mine whether a variable'has a stabie or an explosive time path. Knowledge
of the stability conditions is essential to understanding the recent innovations
in time-series econometrics. The contemporary time-series literature pays
special attention to the issue of stationary versus nonstationary variables.
The stability conditions underlie the conditions for stationarity.
3. Demónstrate how to fmd the soiution to a stochastic difference equation.
There are several different techniques that can be used; each has its own
relative merits. A number of examples are presented to help you understand
the different methods. Try to work through each example carefully. For
extra practice, you should answer the exercises at the end of the chapter.

1. TIME-SERIES MODELS ____________________________

The task facing the modern time-series econometrician is to develop reasonably simple
models capable of forecasting, interpreting, and testing hypotheses concerning
1
 2 CHAPTER 1 DIFFERENCE EQUATIONS

Nolice that the irregular component, while lacking a well-defined pattern, is some-
economic data. The challenge has grown over time; the original use of time-series what predictabte. If you examine the figure closely, you will see that the positive and
analysis was primarily as an aid to forecasting. As such, a methodology was devel- negative valúes occur in runs; the occurrence of a large valué in any period tends to be
oped to decompose a series into a trend, a seasonal, a cyclical, and an irregular com- followed by another large valué. Short-run forecasts willrmake use of this positive cor-
ponent. Uncovering the dynamic path of a series improves forecast accuracy because relation in the irregular component. Over the efiííre sp&Chowever, the irregular com-
each of the predictable components can be extrapolated into the future. ponent exhibits a tendency to revert to zero. As shown in the lower part, the projection
Suppose you observe the fífíy data points shown in Figure 1.1 and are interested of the irregular component past period 50 rapidly decays toward zero. The overall fore-
in forecasting the subsequent valúes. Using the time-series methods discussed in the cast, shown in the top part of the figure, is the sum of each forecasted component.
next several chapters, it is possible to decompose this series into the trend, seasonal,
The general methodology used to make such forecasts entails finding the equa-
and irregular components shown in the lower panel of the figure. As you can see, the
tion ofmotion driving a stochastic process and using that equation to predict subse-
• trend changes the mean of the series, and the seasonal component imparts a regular
quent outcomes. Let y, denote the valué of a data point at period /; if we use this
cyclical pattern with peaks occurring every twelve units of time. In practice, the trend
and seasonal components will not be the simplistic deterministic functions shown in notation, the example in Figure 1.1 assumes we observed y\ through y$§. For t — \ to
this figure. With ecohomic data, it is typical to find that a series contains stochastic 50, the equations of motion used to construct components of the yt series are
elements in the trend, seasonal, and irregular components. For the time being, it is Trend: Tt = 1 + 0.U
f wise to sidestep these complications so that the projection of the trend and seasonal Seasonal: St — 1.6 sin(/7r/6)
components into periods 51 and beyond is straightforward. Irregular: It = 0.7 /M + et
where: T( — valué of the trend component in period / S{ =
valué of the seasonal component in t It = the
valué of the irregular component in t £t — a puré
12 r—j | i | | j j | | j I *] J | | | |
random disturbance in t
_ Observed data .* Forecast ^ __
10
Thus, the irregular disturbance in t is 70 percent of the previous period's irregular dis-
turbance plus a random disturbance term.

:: AJV\/\/ V Each of these three equations is a type of difference equation. In its most general
form^a^difference equation expresses the valué of a variable as a function of its own
lagged valúes, time, and other variables. The trend and seasonal t(erms are both
functions of time and the irregular term is a function of its own lagged valué and of
the stochastic variable -ey/The reason for introducing this set of equations is to make
the point that time-series econometricst is concerned with the estimation of difference
o I i i i i V i i i i i I i i i i i i equations containing stochastic components7The time-series econometrician may
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 estímate the properties of a single series or a vector containing many interdependent
series. Both univariate and multivariate forecasting methods are presented iñ the text,
10 _j | I ^ I j | I j I | | | | J_J j Chapter 2 shows how to estímate the irregular part of a series. Chapter 3 considers
8 - ------ Trend ^^^^^ - estimating the variance when the data exhibit periods of volatility and tranquility.
---- Seasona! ^^«~^^ Estimation of the trend is considered in Chapter 4, which focuses on the issue of
6 ~ ------ I r r e g u l a r ^^^^ Forecasts
whether the trend is deterministic or stochastic. Chapter 5 discusses the properties of
a vector of stochastic difference equations, and Chapter 6 is concerned with the esti-
mation^of trends in a multivariate model.
AltrVóiign forecasting has always been the mainstay of time-series analysis, the
growing importance of economic dynamics has generated new uses for time-series
-2 - analysis. Many economic theories have natural representations as stochastic difference
_ 4 U __ I ---- 1 -- 1 --- 1 _ 1 __ I __ I __I __ l i l i __ L_J __ I 1 equations. Moreover, many of these models have testable implications concern-ing the
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 time path of a key economic variable. Consider the following three examples 1. The
FIGURE 1.1 HypotheticalTime Series Random Walk Hypothesis: In its simplest form, the random walk model suggests that
day-to-day changes in the price of a stock should have
 TIME-SERIES MODELS 5

4. CHAPTER 1 DIFFERENCE EQUATIONS

Equation (1.3) is a structural equation since it expresses the endogenous
variable it as being dependent on the current realization of another
a mean valué of zero.^After all, if it is known thal a capital gain can be made by
endogenous variable, cv A reduced-form equation is one expressing the valué
buying a'sííafe on day ( and selling it for an expected profit the very next day, of a variable in terms of its own lags, lags of other endogenous variables,
efficient speculation will drive up the current price. Similarly, rícfbne will want current and past valúes of exogenous variables, and disturbance terms. As
to hold a stock if it is expected to deprecíate. Formally, the model asserts that the fonmilated, the consumption function is already in reduced form; current
price of a stock should evolve accord-ing to the stochastic difference equation consumption depends only on lagged income and the current valué of the
>',+ 1 = y, + eí+l or stochastic disturbance term ect. Investment is not in reduced form because it
depends on current period consumption.
where: yf — the price of a share of stock on day / To derive a reduced-form equation for investment, substitute (1.2) into the
investment equation to obtain
!
£Í+I = a random disturbance term that has an expected valué of zero i i, = f^oy t _ x + e c t - c t __ x ] + e i t =
Now consider the more general stochastic difference equation: a(3yt_x-0ct_x+fect + eit
Notice that the reduced-form equation for investment is not unique.
You can lag (1.2) one period to obtain: ct_x — oy¡_2 + £ct-\- Using this
The random walk hypothesis requires the testable restriction: QQ = ax = 0. expression, the reduced-form investment equation can also be written as
Rejecting this restriction is equivalent to rejecting the theory. Given the it = a/3yt_ x - /Xoyt_ 2 + £ c/-i) + fect + £it
information available in period t, the theory also requires that the mean of £:/+] be = affy t _ x -JV 2 ) + A^c/ ~ £ ct-\) + £ it O-4)
equal to zero; evidence that £"/+]is predictable invalidates the random walk Similarly, a reduced-form equation for GDP can be obtained by substi-
hypothesis. Again, the appropriate estimation of a single equation model is
considered in Chapters 2 through 4. tuting (1.2) and (1.4) into (1.1):
2. Reduced-Form and Structural Equations: Often it is useful to collapse a y i = °yt-\ + ¿a + °&yt-\ -yt-i) + A^ - %-i) + £ u
system of difference equations into sepárate single-equation models. To = o(l+^y M - Q#V2 + {\+fte ct + eit-(3e ct_x (1.5)
illustrate the key issues involved, consider a stochastic versión of Samuelson's Equation (1.5) is a univariate reduced-form equation;^ is expressed
(1939) classic model: solely as a function of its own lags and disturbance terms r A univariate
model is particularly usefui for forecasting since it eriabíes you to predict a
& = €, + *, -0 (1.1) series based solely on its own current and past realizations. It is possible to
^=^Vi + ^/ 0<a<l (1.2) estimate (1.5) using the univariate time-series techniques explained in
it = ftCf-Cí_]) + £¡r (3>o {L3) Chapters 2 through 4. Once you have obtained estimates of a and /?, it is
where y^ cp and it denote real GDP, consumption, and investment in time straightforward to use the observed valúes ofj^ through yt to predict all
period t, respectively. In this Keynesian modela yp ct, and it are endoge- future valúes in the series {\.e.yyí+x,yí+2, ...)•
nous variables. The previous period's GDP and consumption, V/_i and ct_x, Chapter 5 considers the estimation of multivariate models when all
f are called predetermined or lagged endogenous variables. The terms ect variables are treated as jointly endogenous. The chapter also discusses the
and eit are zero mean random disturbances for consumption and invest- restrictions needed to recover (Le., identify) the structural model from the
ment, and the coeffieients a and /?are parameters to be estimated. estimated reduced-form model.
The first equation equates aggregate output (GDP) with the sum of 3. Error-Correction: Forward and Spot Prices: Certain commodities and
consumption and investment spending. The second equation asserts that financial instruments can be bought and sold on the spot market (for
consumption spending is proportional to the previous period's GDP plus a immediate delivery) or for delivery at some specifíed future date. For
random disturbance term. The third equation illustrates the accelerator example, suppose that the price of a particular foreign currency on the spot
principie. Investment spending is proportional to the change in con- market is st dollars and that the price of the currency for delivery one
sumption; the idea is that growth in consumption necessitates new period into the future is^ dollars. Now, consider a speculator who pur-
investment spending. The error terms ec! and eit represent the portions of chased forward currency at the price ft dollars per unit. At the beginning of
consumption and investment not explained by the behavioral equations of period / + 1, the speculator receives the currency and pays/, dollars per unit
the model.
 6 CHAPTER 1 DIFFERENCE EQUATIONS DIFFERENCE EQUATIONS ANDTHEIR SOLUTIONS 7

received. Since spot foreign exehange can be sold at .v /+1, the speculalor can the valué of the function when evaluated at / = /* + h minus the valué of the function
eam a profit (or loss) of 5" /+] -f¡ per unit transacted. evaluated at /*:
The Unbiased Forward Rate (UFR) hypothesis asserts that expecled AV/H/I = /(/H/Í)-/(/*)
profíts from such speculative behavior should be zero. Formally, the
hypothesis posits the following relationship between forward and spot = »+*-» (1.9)
exchange rates: Differential calculus allows the change in the independent variable (i.e., the term
s t+\ h) to approach zero. Since most economic data is collected over discrete periods,
=ft + ^f+i (1-6)
however, it is more useful to allow the length of the time period to be greater than
where £,+1 has a mean valué of zero from the pgrspective of time period /. In zero. Using difYerence equations, we normalize units so that h represents a unit
(1.6), the forward rate in / is an uiíBiased estímate of the spot rate in H -l. change in / (i.e., h = 1) and consider the sequence of equally spaced valúes of the
Thus, suppose you collected data on the two rates and estimated the independent variable. Without any loss of generality, we can always drop the asterisk
regression on /*. We can then form the first differences:
s
t+\ ** Q^O4" a\ft + e t+l
If you were able to conclude that OQ — 0, Q-J = 1, and that the regression
A>V H ^./C+l) -Á0 = V/+1 -y t &yt+i
residuals eí+l have a mean valué of zero from the perspective of time period
=/^2) -/f+i) = yt+2 -yt+\
/, the UFR hypothesis could be maintained.
The spot and forward markets are said to be in long-run equilibrium Often it will be convenient to express the entire sequence of valúes {... yt-2^yt~\->
when £t+i — 0. Whenever sf+i turns out to differ fromyj, some sort of yf9y(+],y(+2i •• •} as ÍV/}• We can tnen refer to anv particular valué in the sequence as yt. Uniess
adjustment must occur to restore the equilibrium in the subsequent period. specified, the index / runs from -oo to +00. In time-series econometric models, we
Consider the adjustment process use / to represent "time" and h to represent the length of a time period. Thus, v, and
st+2
j>,+1 might represent the realizations of the {yt} sequence in the first and second
= sr+] - «Or+1 -/J + es*2 a>0 (1 -7) quarters of 2004, respectively.
/ r+1 =/ / + /?[ 5f+1 -//] + 9 + i /?>0 (1.8) In the same way we can form the second difference as the change in the first dif-
where esf+2 and £^+1 both have a mean valué of zero. ference. Consider
Equations (1.7) and (1.8) illustrate the type of simultaneous adjustment A2¿',= A(A>y) = &(yt- yt-.\) ^ {yt~ yt-\) - (yt-\ -y^^yt-^yt-x +yt-2
mechanism considered in Chapter 6. This dynamic model is called an error- A2.>Vf, = A(Av/+1) = A(v #+1 -yt ) = (yt+x -yt) - (yt-yt-\)= yt+\ ~ 2^/ +7/-1
correction model because the movement of the variables in any period is
The /?th difTerence (A") is defined analogously. At this point, we risk taking the
related to the previous period's gap from long-run equilibrium. If the spot rate
theory of difference equations too far. As you will see, the need to use second differ-
s^j turns out to equal the forward rate./J, (1.7) and (1.8) state that the spot
ences rarely arises in time-series analysis. It is safe to say that third and higher-order
rate and forward rates are expected to'remain unchanged. If there is a positive
differences are never used in applied work.
gap between the spot and forward rates so that s/+j — ft > 0, (1.7) and (1.8)
lead to the prediction that the spot rate will fall and the forward rate will rise. Since this text considers linear time-series methods, it is possible to examine
only the special case of an /?th-order linear difference equation with constant coeffi-
cients. The form for this special type of difference equation is given by
2, DIFFERENCE EQUATiONS ANDTHEIR SOLUTIONS n
y, = ao + J2 aí yt-i + xt 0 •10)
Althoughmany of the ideas in the previous section were probably familiar to you, it is 1=1
necessary to formalize some of the concepts used. In this section, we will examine the
The order of the difference equation is given by the valué of n. The equation is lin-
type of difference equation used in econometric analysis and make explicit what it
ear because all valúes of the dependent variable are raised to the first power. Economic
means to "solve" such equations. To begin our examination of difference equations, •
theory may díctate instances in which the various a¡ are functions of variables within the
consider the function y =./(/). If we evalúate the function when the independent var iable
economy. However, as long as they do not depend on any of the valúes of yt or xt, we
t takes on the specific valué /*, we get a specifíc valué for the dependent variable called
can regard them as parameters. The term xt is called the forcing process. The form of
>',*. Formally, y,* -flít*). Using this same notation, y,*+h represents the valué of y when
the forcing process can be very general; xt can be any function of time, current and
/ takes on the specific valué t* + h. The first difTerence of v is defined as
lagged valúes of other variables, and/or stochastic disturbances. From an appropriate
choice of the forcing process, we can obtain a wide variety of important macroeconomic
 SOLUTION BY ITERATION 9
8 CHAPTER 1 DIFFERENCE EQUATIONS

models. Re-examine equation (1.5), the reduced-form equation for GDP. This equation Another useful example is provided by the irregular term shown in Figure 1.1*
recall that the equation for this expression is: /; = 0.7/,^ + er You can verify that the
is a second-order diñerence equation since y, depends oni',.2. The forcing process is the solution to this first-order equation is
expression (\+j3)ect + q, - 0scf-\- You will note that (1.5) has no intercept term corre-
sponding to the expression a0 in (1.10). /i = f>7)V, (1.13)
An important special case for the {x,} sequence is i=0
oo Since ( 1 .1 3 ) holds for all time periods, the valué of the irregular component in
/ — 1 is given by
oo
i=0
/r-i=E(°- 7 )Vi-/ (i.i4)
where the f3¡ are constants (some of which can equal zero) and the individual elements 1=0
of the sequence {et} are not functions of the yv At this point it is useful to allow the Now substitute (1.13) and (1.14) into lt ~ 0.7/^j + et to obtain
{£,}' sequence to be nothing more than a sequence of unspecified exogenous vari-
ables. For example, let {£,} be a random error term and set /% = 1 and /3X = /% = ... = £/+0.7£|L.I+(0.7)2ff/.2+(0.7)3ff|U.3 + ...
0; in this case, (1.10) becomes the autoregression equation: = 0.7[^_!+0.7£-/_2+(0.7)2-_3+(0.7)3£^H- ...] + et (1.15)
The two sides of (1.15) are identical; this proves that (1.13) is a solution to the
yt = ao + a\yt-\ + a2yt-2+ • • • + ¿W-» + £t first-order stochastic difference equation J¡ = 0.1It_x + st. Be aware of the distinction
Let n = 1, a0 = 0 and ax = 1 to obtain the random walk model. Notice that equa- between reduced-form equations and solutions. Since lt ~ 0.7Jt_j + et holds for all val-
tion (1.10) can be written in terms of the difference operator (A). Subtracting y,_x úes of/, it follows that /,_! = 0.7/;_2 + et_x. Combining these two equations yields
from (1.10), we obtain
n
/, = 0.7[0.7It_2 + et_{\ + ^=0.49/^2 + 0.1et_x + et (1.16)
+ x Equation (1.16) is a reduced-form equation since it expresses lt in terms of its
yt -y,_i = a 0 + (a x -l);y,_i + ][><>'>-<' '
i= 2
own lags and disturbance terms. However, (1.16) does not qualify as a solution
because it contains the "unknown" valué of /,_2. To qualify as a solution, (1.16) must
or defming 7= (ax~l), we get express It in terms of the elements xt, í, and any given initial conditions.
n
Ayt = a0 + W-i +]Cfl/^-'' + *' (L11)
3. SOLUTiOSM BY ITERATION ^-..-,> _______
1= 2
The solution given by (1.13) was simply postulated. The remaining portions of this
Clearly, equation (1.11) is simply a modifíed versión of (1.10). chapter develop the methods you can use to obtain such solutions. Each method has
A solution to a difference equation expresses the valué of yt as a function of the its own merits; knowing the most appropriate to use in a particular circumstance is a
elements of the {x,} sequence and / (and possibly some given valúes of the {yt} skill that comes only with practice. This section develops the method of iteration.
sequence called initial conditions). Examining (1.11) makes it clear that there is a Although iteration is the most cumbersome and time-intensive method, most people
strong analogy to integral calculus, where the problern is to find a primitive function find it to be very intuitive^
from a given derivative. We seek to fmd the primitive function fit), given an equa- lf the valué ofy in some specifíc period is known, a direct method of solution is
tion expressed in the form of (1.10) or (1.11). Notice that a solution is a function to itérate forward from that period to obtain the subsequent time path of the entire y
rather than a number. The key property of a solution is that it satisfies the difference sequence. Refer to this known valué ofy as the initial condition or the valué of y in
equation for all permissible valúes of / and {*,}. Thus, the substitution of a solution time period 0 (denoted by v0). ít is easiest to illustrate the iterative technique using
into the difference equation must result in an identity. For example, consider the sim- the first-order difTerence equationg
ple difference equation Ay, = 2 (or y, = v,_i + 2). You can easily verify that a solu-
tion to this difference equation is y, = 2t + c, where c is any arbitrary constant. By yt = ao + «i y,-\ + £/ (i.i7)
defmition, if 2/ + c is a solution, it must hold for all permissible valúes of f. Thus for Given the valué of VQ, it follows that vj will be given by
period r-l,j;r-i = 2(/-l) + c. Now substitute the solution into the difference equation y\ = ao + a0'o + e\ In the
to form same way, y? must be '
2/ + c = 2(í- l) + c + 2 (1.12) y-} — c7y + a x y\ + £"->

It is straightforward to carr>' out the algebra and verify that (1.12) is an identity. = a0 + ax[aQ + axy0+ e}] + e{ = a0 +
This simple example also illustrates that the solution to a difference equation need not a0 ax + (ax )2 y0 + ax sx + s2
be unique; there is a solution for any arbitrary valué of c.
 SOLUTION BY ITERATION 11
10 CHAPTER 1 DIFFERENCE EQUATIONS

identity. However, (1.21) is not a unique solution. For any arbitrary valué ofA, a solu-
Continuing the process in'brder to fínd y3, we obtain tion to (1. 17) is given by
y3 = a0 + a}y2 + ^3 = °oíl + a\ + (^l)2] + (^i)33;o + ^l2^ + ^1*2 + £3 You can
easily verify that for all / > 0, repeated iteration yields y, = A a\ + ci0 /(I - a{ ) + Y2a\ st_¡ (i ¿2)
t-\ í-\ /=0

y, = aoX>i + a\ y0 + £>{ ^-' (1J8> To verify that^br any arbitraiy valué of A (1.22) is a solution, substitute (1.22)
i=0 ' 1=0 into (1.17) to obtain
ou oo
Equation (1.18) is a solution to (1.17) since it expresses^ as a function of/, the «0 /(l-«,) + Aa\ +Y,a\£t-i = a0 +«l[«o/( 1"«l) + M~~ ¡ +J2 a\ £t-\-il + £t
forcing process xt = ^(arfe^, and the known valué of v0. As an exercise, it is useful i=0 /=0
to show that iteration from .y, back to^0 yields exactly the formula given by (1.18).
Since yt = a0 + axyt_x + €„ it follows that Since the two sides are identical, (1.22) is necessarily a solution to (1.17).
+ e t-\\ £t
yt = aQ + al [a0 + tf ,j>,__2 + Reconciling the Two Iterative Methods
= ao(\+ax) + ax£t_x + £t + tfi2l>o + ¿W-3 + ^-2]
Given the iterative solution (1.22), suppose that you are now given an initial condition
Continuing the iteration back to period 0 yields equation (1.18).
concerning the valué of y in the arbitrary period t0. It is straightforward to show that
we can impose the initial condition on (1.22) to yield the same solution as (1.18). Since
Iteration without an Initial Condition (1.22) must be valid for all periods (including t0), when / = 0, it must be true that
oo
Suppose you were not given the initial condition for>>0. The solution given by (1.18) yQ = A + aQ /(\-a{) + Y¿a\£-i so that
i=0
would ¿o longer be appropriate because the valué of y0 is an unknown. You could not oo
select this initial valué of y and itérate forward, ñor could you itérate backward from
yt and simply choose to stop at t = t0. Thus, suppose we continued to itérate backward i=0
by substituting a0 + axy_i + £Q forj^o in (1.18): Since y Q is given, we can view (1.23) as the valué of A that renders (1.22) a solu-
i-\ t-\ tion to (1.17) given the initial condition. Henee, the presence of the initial condition
yt = aoYlai + °i [¿2o + ai J-i + £°^ + 5^fli et-¡ eliminates the arbitrariness of A. Substituting this valué ofA into (1.22) yields
1=0 /=0
£ +1 = / 1 oo / 1 £í °° 24
= «0¿fll + YA t-i + ^l 3-l 0-19) ^r LvO-«O ( -«l)-^k-/]«í + «O ( - l) + X]«l^-/ O- )
i=0 1=0 /=0 /=0
Continuing to itérate backward another m periods, we obtain Simplifícation of (1.24) results in
r+m t+m r-l
a
yt = oYl"\ + E^í /-/ +flí e +m+1
y-m-i (1.20) 3'/ = f>?()-«o/(l-«i)]<3Í + ao/(l-«i) + X]<3Í£"/-/ (1-25)
i=0 /=0 /=0

Now examine the pattern emerging from (1.19) and (1.20). If 1^1 < 1, the term You should take a moment to verify that (1.25) is identical to (1.18).
«]/+m+1 approaches zero as m approaches infinity. Also, the infinite sum [1 + ax + (aj)2
+ ...] converges to 1/(1 — ax). Thus, if we temporarily assume that \ax\ < 1, after Nonconvergent Sequences
continual substitution, (1.20) can be written as
Given that |o,| < 1, (1.21) is the Hmiting valué of (1.20) as m grows infinitely large.
What happens to the solution in other circumstances? If \a x \ > \, it is not possible to
/=0
move from (1.20) to (1.21) because the expression |£?1|/+m grows infinitely large as
You should take a few minutes to convince yourself that (1.21) is a solution to t+m approaches infinity.1 However, if there is an initial condition, there is no need to
the original difference equation ( 1 . 1 7 ) ; substitution of ( 1 . 2 1 ) into ( 1 . 1 7 ) yields an
12 CHAPTER 1 DIFFERENCE EQUATIONS

obtain the infinite summation. Sinxply select the initial condition y0 and itérate for-
ward; the result will be (1.18):
r-l / -I

,=0 í=0
Although the successive valúes of the {yt} sequence will become progressively larger
in absolute valué, all valúes in the series will be finite. A very interesting case arises
if ax = 1. Rewrite (1.17) as:

or
Aj>, = a0 + £/
As you should verify by iterating fromy, back to^ * solution to this equation is¿
t

After a moment's reflection, the form of the solution is quite intuitive. In every
period U the valué of yt changes by aQ + £, units. After / periods, there are / such
changes; henee, the total change is ta0 plus the / values of the {et} sequenee. Notice •
that the solution contains summation of all disturbances from ex through ev Thus,
when ax = 1 each disturbance has a permanent non-decaying effect on the valué of y
You should compare this result to the solution found in (1.21). For the case m
which \ax\ < 1, |*i |' is a decreasing function of t so that the efTects of past distur-
bances become successively smaller over time.
The importance of the magnitude of ax is illustrated in Figure 1.2. Twenty-five
random numbers with a theoretical mean equal to zero were computer -generated
and denoted by ex through s25. Then the valué of y0 was set equal to unity and the
next twenty-five values of the {v,} sequence were constructed using the formula
= 0 9y + e The result is shown by the thin Une in Panel (a) of Figure 1.2. If you
súbstitute tf0 = 0 and a} = 0.9 into (1.18), you will see that the time path of {yt} con-
sists of P*>o parts. The first part, 0.9', is shown by the slowly decaymg thick line in
the panel. This term dominates the solution for relatively small values of u The influ-
ence of the random part is shown by the difference between the thin and the thick line;
you can see that the fírst several values of {et} are negative. As t increases, the mflu-
ence of the random component becomes more pronounced. FIGURE 1.2 Convergent and Nonconvergent Sequences
Using the previously drawn random numbers, we again set y0 equal to unity and
a second sequence was constructed using the formula yt = 0.53^! + ev This second smaller valué of a l means thaí the past realizations of £ t_¡ have a smaller influence on
sequence is shown by the thin line in Panel (b) of Figure 1.2. The influence of the the current valué of y t.
expression 0.5' is shown by the rapidly decaying thick line. Again, as t increases, the Simulating a third sequence with a¡ = -0.5 yields the thin line shown in Panel (c).
random portion of the solution becomes more dominant in the time path of {yt}. The oscillations are due to the negative valué of O j . The expression (—0.5)', shown by
When we compare the first two panels, it is clear that reducing the magnitude of |*,| the thick line, is positive when / is even and negative when / is odd. Since \ a ^ \ < 1, the
increases the rate of convergence. Moreover, the discrepancies between the simulated oscillations are dampened.
values of v, and the thick line are less pronounced in the second panel. As you can see The next three panels in Figure 1.2 all show nonconvergent sequences. Each uses
in (1.18), each valué of £-,_,- enters the solution for v, with a coefficient of {ax)¡. The the initial condition y 0 = 1 and the same twenty-five values of {s t } used in the other
14 CHAPTER 1 DIFFERENCE EQUATIONS AN ALTERNATIVE SOLUTION METHODOLOGY 15

simulations. The thin line in Panel (d) shows the time path ofyt=yt_\ + ev Since each Now consider (1.17) in its entirety. In the last seciion, you confirmed that (1.21)
valué of e¡ has an expected valué of zero, Panel (d) illustrates a random-walk process. is a valid solution to (1.17). Equation (1.21) is called a particular solution to the dif-
Here Ay{ - et so that the change inyt is random. The nonconvergence is shown by the ierence equation; all such particular solutions will be denoted by the term yrP. The
tendency of {yt} to meander. In Panel (e), the thick line representing the explosive term "particular" stems from the fact that a solution to a difference equation may not
expression (1.2)' dominates the random portion of the {yt} sequence. Also notice that be unique; henee, (1.21) is just one particular solution out of the many possibilities.
the discrepancy between the simulated {y,} sequence and the thick line widens as / In moving to (1.22) you verified that the particular solution was not unique. The
increases. The reason is that past valúes of £t__¡ enter the solution for yf with the coef- homogeneous solution Aa\ plus the particular solution given by (1.21) constituted the
fícient (1.2)1'. As / increases, the importance of these previous discrepancies becomes complete solution to (1.17). The general solution to a difference equation is defined
increasingly important. Similarly, setting a^ = —1.2 results in the exploding oscilla- to be a particular solution plus all homogeneous solutions. Once the general solution
tions shown in the lower-right panel of the figure. The valué (—1.2)' is positive for is obtained, the arbitrary constant A can be eliminated by imposing an initial condi-
even valúes of t and negative for odd valúes of t. tion for }>Q.

4- AN ALTERNAT1VE SOLUTION METHODOLOGY The Solution Methodology

Solution by the iterative method breaks down in higher-order equations. The alge- The results of the first-order case are directly applicable to the wth-order equation
braic complexity quickly overwhelms any reasonable attempt to find a solution. given by (1.10). In this general case, it will be more difficult to find the particular
Fortunately, there are several alternative solution techniques that can be helpñil in solution and there will be n distinct homogeneous solutions. Nevertheless, the solu-
solving the wth-order equation given by (1.10). If we use the principie that you tion methodology will always entail the following four steps:
should learn to walk before you learn to run, it is best to step through the first-order STEP 1: Form the homogeneous equation and find all n homogeneous solutions;
equation given by (1.17). Although you will be covering some familiar ground, the STEP 2: Find a particular solution;
first-order case illustrates the general methodology extremely well. To split the pro- STEP 3: Obtain the general solution as the sum of the particular solution and a lin-
cedure into its component parts, consider only the homogeneous portion of (1.17): 3 ear combination of all homogeneous solutions;
STEP 4: Elimínate the arbitrary constant(s) by imposing the initial condition(s) on
the general solution.
The solution to this homogeneous equation is called the homogeneous solution;
at times it will be useful to denote the homogeneous solution by the expression vf. Before we address the various techniques that can be used to obtain homogeneous
Obviously, the trivial solution^ = yt_i - ... =0 satisfies (1.27). However, this solu- and particular solutions, it is worthwhile to illustrate the methodology using the equation:
tion is not unique. By setting a0 and all valúes of {£>} equal to zero, (1.18) becomes F/ = 0.9>v_1-0.2v/_2 + 3 (1.28)
yt = a\yQ. Henee, y, = a[y0 must be a solution to (1.27). Yet, even this solution does Clearly, this second-order equation is in the form of (1.10) with ¿/0 = 3, a\ — 0.9,
not constitute the full set of solutions. It is easy to verify that the expression a\ mul- a2 = -0.2, and xt = 0. Beginning with the first of the four steps, form the homoge-
tiplied by any arbitrary constant A satisfies (1.27). Simply substitute yr = Aa \ and yt_] = neous equation:
Aa\~x into (1.27) to obtain
yt - 0.9yM + 0.2v,_2 = 0 (1.29)
Aa\ = a x Aa[~ x
In the first-order case of (1.17), the homogeneous solution was Aa{. Section
Since a\ = a^a\~\ it follows that yt = Aa\ also solves (1.27). With the aid of the 6 will show you how to find the complete set of homogeneous solutions. For now,
thick lines in Figure 1.2, we can classify the properties of the homogeneous solution it is sufficient to assert that the two homogeneous solutions are: y{t ~ (0.5)' and
as follows: y'ii = (0.4)'. To verify the first solution, note that yl,-\ = (0.5)'" 1 and y,',._ 2 =
1. If \ax | < 1. the expression a\ converges to zero as / approaches infinity. (0.5y~2. Thus, y*{t is a solution if it satisfies
Convergence is direct if 0 < a^ < 1 and oscillatory if —1 <■ a\ < 0. (o.5y - 0.9(0.sy-1 + o.2(o.5y~ 2 = o
2. If \ax | > 1, the homogeneous solution is not stable. If ax > 1, the homoge If we divide by (O.5y~2, the issue is whether
neous solution approaches infinity as t increases. If a} < -1, the homoge
neous solution oseillates explosively. (0.5)2 - 0.9(0.5) + 0.2 = 0
3. If a ] = ], any arbitrary constant A satisfies the homogeneous equation Carrying out the algebra, 0.25 - 0.45 + 0.2 does equal zero so that (0.5)' is a solu-
>'/ ~.v/-i- If ¿?i — —1, the system is meta-stahle: a\ — 1 for even valúes of tion to (1.29). In the same way, it is easy to verify that v?, = (0.4)' is a solution since
/ and -1 for odd valúes of t. (0.4)/._ 0.9(0.4)'-» + 0.2(0.4y~2 = 0
16 CHAPTER 1 DIFFERENCE EQUATIONS THE COBWEB MODEL 17

Divide by (0.4y-2 to obtain (0.4) 2 - 0.9(0.4) + 0.2 = 0.16 - 0.36 + 0.2 = 0. Regrouping terms. we vvant to know if
The second step is to obtain a particular solution; you can easily confírm that the
particular solution jy7 = 10 solves (1.28) as: 10 = 0.9(10) - 0.2(10) + 3. ^i yu ~ í>i«Lví,_,) + <¿2 y'i, - ÍtA2«¡ yí-i) = o
The third step is to combine the particular solution and a linear combination of
both homogeneous solutions to obtain Since A\yhu and A2y2, are sepárate solutions to (1.30), each of the expressions in
brackets is zero. Henee, the linear combination is necessarily a solution to the homo-
yt = AÚQ.Sy + A2(0Ay + 10
geneous equation. This result easily generalizes to all n homogeneous solutions to an
where A\ and A2 are arbitrary constants. /7th-order equation.
For the fourth step, assume you have two initial conditions for the {yt } Finally, the use of Step 3 is appropriate since the sum of any particular solution
sequence. So that we can keep our numbers reasonably round, suppose that^Q =13 and any linear combination ofall homogeneous solutions is also a solution. To prove
anáyi - 11.3. Thus, for periods zero and one, our solution must satisfy this proposition, substitute the sum of the particular and homogeneous solutions into
n=Al+A2+\0 11.3= .0-10) to obtain
^(0.5)+ /Í2(O.4)+ 10. y? + y'! = «o + í> (vf-/ + //-,) + * (i.33)
Solving simultaneously for A j and A2, you should find A\~ \ and A2 — 2. Henee, /=i
the solution is Recombining the terms in (1.33), we want to know if

jy= (0.5)' + 2(0.4)' + 10 b'f-«o -í>>'f-,-^] + tyf-E^Í1-,-] = o (L34)

/=i i=i

Generalizing the Method Since y f solves (1.10), the expression in the first bracket of (1.34) is zero. Since
y1} solves the homogeneous equation, the expression in the second bracket is zero.
To show that this method is applicable to higher-order equations, consider the homo- Thus, (1.34) is an identity; the sum of the homogeneous and particular solutions
geneous part of (1.10): solves (1.10).
n
*=&?,-,• (i-30) 5, THE COBWEB MODEL ________________________
1=1

As shown in Section 6, there are n homogeneous solutions that satisfy (1.30). For An interesting way to illustrate the methodology outlined in the previous section is to
now, it is sufficient to demónstrate the following proposition: Ify*¡ is a homogeneous consider a stochastic versión of the traditional cobweb model. Since the model was
solution to (1.30), Ayht is also a solution for any arbitrary constant A. By assumption, originally developed to explain the volatility in agricultural prices, let the market for
yht solves the homogeneous equation so that a product—say wheat—be represented by
dt = a-~pt 7>0 (1.35)
y?=5>íy?-,- o.3i)
St = b + 0px+sx /3>0 (1.36)
The expression Ayht is also a solution if:
st = dt (1.37)
Ay^f^atAy^ (1.32) where: dt — demand for wheat in period / s¡ ~ supply of wheat
i=l
in / pt ~ market price of wheat in / p* = price that farmers
We know (1.32) is satisfied because dividing each term by A yields (1.31). Now expect to prevail at / £t = a zero mean stochastic supply shock
suppose that there are two sepárate solutions to the homogeneous equation denoted and parameters a, /?, 7, and 6 uve all positive such that a > ¿>.4
by y\¡ and y\r It is straightforward to show that for any two constants Ax and A2, the
linear combination A\y\x + A^y\x is also a solution to the homogeneous equation. If The nature of the model is such that consumers buy as much wheat as is desired
A\y\t + ^2>;2Í is a solution to (1.30), it must satisfy at the market clearing price pr At planting time, farmers do not know the price pre-
vailing at harvest time; they base their supply decisión on the expected price (p*). The
• A
\ yhu + A2 y ii = «i (A yf,-i + A2 y^-i*
+«2 Wi ykit-2 + A2 yh2t-2) + .» + fln (A, yhu_n + A2 yh2í_n)
18 CHAPTER1 DIFFERENCE EQUATIONS
THE COBWEB MODEL 19

actual quantity produced depends on the planned quantity b + ¡3p*t plus a random sup- willing to buy quantity sf+l only if the price falls to that indicated by pf+l (see point_3 in
ply shock et. Once the product is harvested, market equilibrium requires that the the figure). The next period begins with farmers expecting to be at point 4. The
quantity supplied equal the quantity demanded. Unlike the actual market for wheat, process continually repeats until the equilibrium point E is attained.
the model does not allow for the possibility of storage. The essence of the cobweb As drawn. Figure 1.3 suggests that the market will always converge to the long-
model is that farmers fomi their expectations in a naive fashion; let farmers use last run equilibrium point. This result does not hold for all demand and supply curves. To
year's price as the expected market price formally derive the stability condition, combine (1.35) through (1.38) to obtain
P*,=P,-x d-38) ¿ + 0Pt-\ + £t = °-lPt or
Point E in Figure 1.3 represents the long-run equilibrium price and quantity com- Pt = i~ffÍ)Pt~\ + O ~ W7- ¿th (1 39)
bination. Note that the equilibrium concept in this stochastic model di ñers from that
Clearly, (1.39) is a stochastic first-order linear difference equation with constant
of the traditional cobweb model. If the system is stable, successive prices will tendió coefficients. To obtain the general solution, proceed using the four steps Usted at the
converge to point E. However, the nature of the stochastic equilibrium is such that the end of the last section:
ever-present supply shocks prevent the system from remaining at E. Nevertheless, it is
useful to solve for the long-run price. If we set all valúes of the {£,} sequence equal to 1. Form the homogeneous equation: pt = (r$7)Pt-\~ m ^e next section you
zero, setpf -pt_x — ... — p, and equate supply and demand, the long-run equilibrium will learn how to find the solution(s) to a homogeneous equation. For now,
price is given by p — {a — b)/(y^-0). Similarly, the equilibrium quantity (s) is given by it is sufficient to verify that the homogeneous solution is
s = (a/?+<76)/(7+/?).
To understand the dynamics of the system, suppose that farmers in t plan to pro-
duce the equilibrium quantity s. However, let there be a negative supply shock such where A is an arbitrary constant.
that the actual quantity produced turns out to be st As shown by point 1 in Figure 1.3, 2. If the ratio J3í-y\s less than unity, you can itérate (1.39) backward fromp, to
consumers are willing to pay pt for the quantity st; henee, market equilibrium in t verify that the particular solution for the price is
oceurs at point 1. Updating one period allows us to see the main result of the cobweb
model. For simplicity, assume that all subsequent valúes of the suppiy shock are zero ^ = ^-^£(-/^)V,- (1-40)
(i.e., eí+i - et+2 = ... = 0). At the beginning of period / + 1, farmers expect the price at If &7> 1, the infinite summation in (1.40) is not convergent. As dís-
harvest time to be the price of the previous period; thus p" +1 = pv Accordingly, they cussed in the last section, it is necessary to impose an initial condition on
produce and market quantity sí+x (see point 2 in the figure); consumers, however, are (1.40) \f/Sfy> 1.
3. The general solution is the sum of the homogeneous and particular solu-
tions; combining these two solutions, the general solution is
n—h 1 _°°.
pí^^-~---J2(-^^Í^~i+Á(-^/^t (1-4D
7+P 7 /=0
4. In (1.41), A is an arbitrary constant that can be eliminated if we know the
price in some initial period. For convenience, let this initial period have a
time subscript of zero. Since the solution must hold for every period,
including period zero, it must be that case that

P0 = ^~ ~1 X>/?/ ?)'■ e-¡ + A (-Í31 7)°

Since (- J$T)° = 1, the valué oí A is given by

20 CHAPTER 1 DIFFERENCE EQUATIONS THE COBWEB MODEL 21

Substituting this solution for A back into (1.41) yields a price above the long-run pricep\ the price in / rises by l/S-units for each unit decline
in current period supply. Of course, this terminology is not specific to the cobweb
model; in terms of the mh-order model given by (1.10), the impact multiplier is the
partial derivative of r; with respect to the partial change in the forcing process. 5
and after simplifícation of the two summations The effeets of the supply shock in / persist into future periods. Updating (1.42)
by one period yields the one-period multiplier:

We can interpret (1.42) in terms of Figure 1.3. In order to focus on the stability of
the system, temporarily assume that all valúes of the {£,} sequence are zero.
Subsequently, we will retuni to a consideration of the effects of supply shocks. If the sys-
tem begins in long-run equilibrium, the initial condition is such thatp0 = (a - 6)/(7+ 0). Point 3 in Figure 1.3 illustrates how the price in / + 1 is affected by the nega-
In this case, inspection of equation (1.42) indicates that pt — (a — b)/(y+ 0). Thus, if we tive supply shock in /. It is straightforward to derive the result that the effects of
begin the process at point E9 the system remains in long-run equilibrium. Instead, sup- the supply shock decay over time. Since 0/y < 1, the absolute valué of dpt /det
pose that the process begins at a price below long-run equilibrium: p0 < (a - ¿)/(7+ 0). exceeds dpi+i/ds t. All of the multipliers can be derived analogously; updating
Equation (1.42) tells us thatpj is (1.42) by two periods:
px = (a - ¿)/(7+ 0) + [Po ~ (fl - ¿V(7 + 0)] {-0¡i)x (1.43) dPí^dst = -{\/i){r0/y9'
< a
Sincepo ( ~ fc)/(7 + 0) and -01 y < 0, it follows thatpj will be above the long- and after n periods:
run equilibrium price (a — b)/(y + 0). In period 2 dpt+n/dst = -(\/y)(-0/y)»
P2 = (°- *V(7+ ft + [ P0 - (fl - fc)/( 7 + #] (-/^7) 2 The time path of all such multipliers is called the impulse response function.
althoughpo < (tf — b)/(y+ /?)> (—0/y)2 is positive; henee, p2 ^s below the long-run equi- This function has many important applications in time-series analysis because it
librium. For the subsequent periods, note that (r-0/y)* will be positive for even valúes shows how the entire time path of a variable is affected by a stochastic shock. Here,
of t and negative for odd valúes of t. Just as we found graphically, the successive val- the impulse function traces the effects of a supply shock in the wheat market. In other
úes of the {p¡} sequence will oscillate above and below the long-run equilibrium economic applications, you may be interested in the time path of a money supply
price. Since {01 y)1 goes to zero if 0 < y and explodes if 0 > 7, the magnitude of 0/y shock or a productivity shock on real GDP.
determines whether the price actually converges toward the long-run equilibrium. If In actuality, the function can be derived without updating (1.42) because it is
0/y< 1, the oscillations will diminish in magnitude, and if 01 y > 1, the oscillations always the case that
will be explosive. Opt +j^ dpt
The economic interpretaron of this stability condition is straightforward. The
slope of the supply curve (i.e., dptlds¡) is l//?and the absolute valué of the slope of
the demand curve [i.e., ~dpt fd(dí)] is 1/7. If the supply curve is steeper than the To find the impulse response function, simply find the partial derivative of (1.42)
demand curve, \I0> Myor 0/y< 1 so that the system is stable. This is precisely the with respect to the various s,¡. These partial derivatives are nothing more than the
case illustrated in Figure 1.3. As an exercise, you should draw a diagram with the coefficients of the {¿>_y-} sequence in (1.42).
demand curve steeper than the supply curve and show that the price oscillates and Each of the three components in (1.42) has a direct economic interpretation. The
diverges from the long-run equilibrium. deterministic portion of the particular solution (a — h)/(y+ 0) is the long-run equi-
Now consider the effects of the supply shocks. The contemporaneous efíect of a librium price; if the síability condition is met, the {pt} sequence tends to converge
supply shock on the price of wheat is the partial derivative ofp, with respect to et\ to this long-run valué. The stochastic component of the particular solution captures
from (1.42): the short-run price adjustments due to the supply shocks. The ultímate decay of the
coefficients of the impulse response function guarantees that the effects of changes
^fl = -L (1.44) in the various s, are of a short-run duration. The third component is the expression
d st 7
{~/ü -))>/! = (-JLJ/^YIPQ ~ (a ~~ bV(l + 0)]- The valué of A is the initial periodos devia-
Equation (1.44) is called the impact multiplier since it shows the impact efíect tion of the price from its long-run equilibrium level. Given that i'J/y< I, the impor-
of a change in et on the price in /. In terms of Figure 1.3, a negative valué of et implies tance of this initial deviation diminishes over time.
22 CHAPTER 1 DIFFERENCE EQUATIONS SOLVING HOMOGENEOUS DIFFERENCE EQUATIONS 23

6. SOLVING HOMOGENEOUS equation denoted by (o,)' and (ex2y. We already know that any linear combi-
DIFFERENCE EQUATIONS _________________________ nation of the two is also a solution. Henee,
Higher-order difTerence equations arise quite naturally in economic analysis. Equation y[l = A, (¿v,)' +A2U*2)'
(1.5)__ the reduced-form GDP equation resulting from Samuel son 's (1939) model—is
an example of a second-order difference equation. Moreover, in time-series economet- It should be clear that if the absolute valué of either ax or a2 exceeds unity,
rics it is quite typical to estímate second and higher-order equations. To begin our the homogeneous solution will explode. Worksheet 1.1 examines two second-
examination of homogeneous solutions, eonsider the second-order equation order equations showing real and distinct characteristic roots. In the first example, yt
= 0.2yt_x + 0.35y,_2, the characteristic roots are shown to be: a{ = 0.7 and o¿2 = -0.5.
y, - axyt_x ~ aiyt-2 = ° (] -45> Henee, the full homogeneous solution isj>í' =A](0.7y + A2(-0.5)í. Since both roots
Given the fíndings in the first-order case, you should suspect that the homogeneous are less than unity in absolute valué, the homogeneous solution is conver-gent. As
solution has the form7? =*Ac¿. Substitution of this trial solution into (1.45) yields you can see in the graph on the bottom left-hand side of Worksheet 1.1,
Aa?-axAah* -a2Ao¿-2 = § (1.46) convergence is not monotonic because of the influence of the expression (-0.5)'.
Clearly, any arbitrary valué of A is satisfactory. If you divide (1.46) by A a'"2, the In the second example,^ = 0.7yt_x + 0.35v^* ^ne worksheet indicates how to
problem is to fínd the valúes of a that satisfy obtain the solution for the two characteristic roots. Given that one characteristic
root is (1.037)', the {yt} sequence explodes. The influence of the negative root (a2 =
Q 2_ a x a-a 2 =0 (1.47)
-0.337) is responsible for the non-monotonicity of the time path. Since (-0.337)'
Solving this quadratic equation—called the characteristic equation—yields quickly approaches zero, the dominant root is the explosive valué 1.037.
two valúes of a, called the characteristic roots. Using the quadratic formula, we fínd
that the twó characteristic roots are
a x ±Jai+4a 2
CASE 2
a},a2= ----------------------
If ax2 + 4a2 = 0, it follows that d = 0 and ax = a2 = a^ll. Henee, a homogeneous
= (ax±y[d)/2 (1.48) solution is ax/2. However, when d = 0, there is a second homogeneous solution
given by /(¿ÍJ/2)'. To demónstrate thatjyf = t(ax I 2)' is a homogeneous solution,
where d is the discriminant [(ax)2 + 4a2]. substitute it into (1.45) to determine whether
Each of these two characteristic roots yields a valid solution for (1.45). Again,
these solutions are not unique. In fact, for any two arbitrary constants Ai and A2, the Hp x ny - fl| [(/-l)(a,/2)'-l] - a 2 [(/-2)( ai /2)'-2] = 0
linear combination Ax(axy + A2(a2)f also solves (1.45). As proof, simply substitute y, Divide by (fl|/2)'-2 and form
= A x(ax)f + A2(a2y into (1.45) to obtain
-t(^/4) + a 2 ]/ + [(af/2) + 2a 2 ] = 0
Ax{axy + A2(a2Y = ax[Ax(ax)f~l + A2íatf-*\ + a2[Ax(axy~2 + A2{a2y~2} Now,
regroup terms as follows: Since we are operating in the circumstance where a\ + Aa2 = 0, each brack-
eted expression is zero; henee, t{ax/2y solves (1.45). Again, for arbitrary con-
y~ l
A x [(a x y ~ o x (a x y~ 2 ]
- a 2 (a x y~ 2 }
+ A 2 [(a 2 y - a^a^ - a 2 {a 2 = 0 Since ax and a-
stants A j and A2, the complete homogeneous solution is
? each solve (1.45), both terms in brackets must equal zero. As such, the complete
y h t =/\ 1 (í7 1 /2) / +A 2 r(fl 1 /2) í
homogeneous solution in the second-order case is
y* =A x (a x y +A2 (a 2? Clearly, the system is explosive if \ a x \ > 2. If l^l < 2, the term Ax(ax/2y
converges, but you might think that the effect of the term t{ax/2y is ambigu-
Without knowing the specifíc valúes of ax and a2, we cannot fínd the two char- ous [since the diminishing (ax/2y is multiplied by /]. This ambiguity is correct
acteristic roots ax and a2. Nevertheless, it is possible to characterize the nature of the in the limited sense that the behavior of the homogeneous solution is not
solution; three possible cases are dependent on the valué of the discriminant d. monotonic. As illustrated in Figure 1.4 for ax/2 - 0.95, 0.9, and -0.9, as long
as |¿7j| < 2, lim[ t{ax/2y ] is necessarily zero as / —•> oo; thus, there is always
CASE 1 convergence. For 0 < ax < 2, the homogeneous solution appears to explode
before ultimately converging to zero. For -2 < ax < 0, the behavior is wildly
If ax2 + Aa2 > 0, d is a real number and there will be two distinct real charac- erratic; the homogeneous solution appears to oscillate explosively before the
teristic roots. Henee, there are two sepárate solutions to the homogeneous oscillations dampen and finally converge to zero.
24 CHAPTER 1 DIFFERENCE EQUATIONS

SECOND-ORDER EQ UA TJONS
Example 1: yt = Q.2,y,_l + 0.35>v_2- Henee: ax = 0.2 and a2 = 0.35
Forni the homogeneous equation: yt - 0.2>v-i — 0.35vr_2 = 0
A check of the discriminant reveáis: d- af + Aa2 so that ¿/ = 1.44. Given that
d > 0, the roots will be real and distinct.
Let the trial solution have the form: y, = o!. Substitute the trial solution into
the homogenous equation to obtain: cí — 0.2ct~^ — 0.35 a'"2 = 0
Divide by af~2 to obtain the characteristic equation: a2 - 0.2 o* - 0.350-= 0
Compute the two characteristic roots:
<*! = 0.5(a! + dm) oti = 0.5(0, - d m)
oc\ =0.7 OQ = -0.5
The homogeneous solution is: Ai(0jy + A2(~Q.Sy. The fírst graph shows
the time path of this solution for the case in which the arbitrary constants
equal unity and / runs from 1 to 20.
Example 2: y, = 0.7>Y-i + 0.35j^_2- Henee: a\ = 0.7 and a2 = 0.35
Form the homogeneous equation: yt — 0.7>v_i - 0.35>>,_2 = 0
A check of the discriminant reveáis: d—a}+ 4a2 so that d = 1.89. Given that
d> 0, the roots will be real and distinct.
Form the characteristic equation cé — 0.7 o7"1 - 0.35 a?~2 = 0
Divide by o!~2 to obtain the characteristic equation: a2 - 0.7a: - 0.35a = 0
Compute the two characteristic roots:
a-i = 0.5(a] + dm) or¿ = 0.5(a! - dm)
aj = 1.037 O2 =-0.337
The homogeneous solution is: A]{\ .037)r + A2(-033iy. The second graph
shows the time path of this solution for the case in which the arbitrary con-
stants equal unity and 1 runs from 1 to 20. CASE 3
If a|2 + 4¿?2 < 0. it follows that d is negative so that the characteristic roots are
imaginary. Since ax2 > 0, imaginar/ roots can oceur only if a2 < 0. Although
this might be hard to interpret directly, if we switch to polar coordinates it is
possible to transform the roots into more easily understood trigonometric func-
tions. The technical details are presented in Appendix 1.1. For now, write the
two characteristic roots as
ft,=(ííj4-[NiQ)/2 c*2=(a{-iyF^d)/2

where / = \/-1.
As shown in Appendix 1.1, you can use de Moivre's theorem to write the
homogeneous solution as
y¡ = l\ r cos( Oí + d2) (1.49)
26 CHAPTER 1 DIFFERENCE EQUATIONS SOLVING HOMOGENEOUS DIFFERENCE EQUATIONS 27

where fl\ and /í> aré'arbitrary constants, r = (-a2)]/2> and the valué of #is cho-sen
so as to satisfy |^EZ£^0££M^yiaiiÍUflEJEE
cos(0 = a )/[2(-^2)l/2] (1.50)
IMAGINARY ROOTS _________________________
The trigonometric functions impart a wavelike pattern to the time path of Example 1 Exampie 2
the homogeneous solution; note that the frequency of the oscillations is deter- y, - 1.6V/-I + 0.9^.2 yt + 06-v/-1 + °'9>V2
mined by 6. Since cos(#) = COS(2TT+ $), the stability condition is determined
solely by the magnitude of r ~ (-¿?2)1/2. If l^l = U the oscillations are of a) Check the discriminant d - (a\)2 + 4a 2
unchanging amplitude; the homogeneous solution is periodic. The oscillations d = (1.6)2 + 4(-0.9) d = (-0.6)2 + 4(-0.9)
will dampen if \a2\ < 1 and explode if \o2\ > 1. = -1.04 =-3.24
Example: It is worthwhile to work through an exercise using an equation with Henee, the roots are imaginary. The homogeneous solution has the form
imaginary roots. The left-hand side of Worksheet 1.2 examines the behavior of
the equation yt — 1.6y,_j — 0.9yf_2. A quick check shows that the discriminant d yf= 0/cos(0t + fo)
is negative so that the characteristie roots are imaginary. If we transform to polar
coordinates, the valué of r is given by (0.9)1/2 = 0.949. From (1.50), cos(#) = where (3\ and fi2 are arbitrary constants.
1.6/(2* 0.949) = 0.843. You can use a trig table or a calculator to show that 0- b) Obtain the valué of r = (-«2)1/2
0.567 (i.e., if cos(0 = 0.843, 0= 0.567). Thus, the homogeneous solution is r=(0.9)
1/2 r = (0.9)l/2
y1; = # (0.949/ cos(0.567/ + /% ) (1.51) = 0.949 = 0.949

The graph on the left-hand side of Worksheet 1.2 sets /3} = 1 and /% = 0 and c) Obtain 6 from cos(0) = ax/\2(~ai) m]
plots the homogeneous solution for t — 1, ..., 30. Example 2 uses the same cos(0) - 1.6/[2(0.9)1/2] cos(^) = -0.6/[2(0.9)1/2]
valué of a2 (henee, r = 0.949) but sets ax = —0.6. Again, the valué of d is nega- = 0.843 =-0.316
tive; however, for this set of calculations, cos(#) = -0.316 so that 0 is 1.89. Given cos(^), use a trig-table to find G
Cornparing the two graphs, you can see that increasing the valué of 6 acts to
^ = 0.567 é?=1.89
increase the frequency of the oscillations.
d) Form the homogeneous solution: y? = P\r* cos(0t+f3 2)
Stability Conditions y? = 0,(0.949)' cos(0.567/ + 02) $ = /?,(0.949)' cos(l .89/ + fc)
For P\ = 1 and j32 = 0. the time Paths of the homogeneous solutions are
The general stability conditions can be summarized using triangle ABC in
Figure 1.5. Are A0B is the boundary between Cases 1 and 3; it is the locus of points
where d = ¿7j2 + Aa2 - 0. The región above A0B corresponds to Case 3 (since d> 0), 1
T 1
"
and the región below A0B corresponds to Case 3 (since d < 0).
In Case 1 (in which the roots are real and distinct), stability requires that the
largest root be less than unity and the smallest root be greater than —1. The largest
characteristie root, a^ — (ü]+\/d)/2, will be less than unity if
ax + (a}2 + 4a2)]/2 < 2 or (ax2 + 4a2y12 <2-ax
Henee, a}2 + 4a2 < 4 - 4a} + ax2 or
-o\ ¿^v> mfWvWVU
a\ + « 2 < ] (1-52)
^1 -1--
The smallest root, a2 = {ax - Vd)/2* will a ] - ( o ] 2 + 4a 2)l/2>-2 or 2 +
a } > (a}2 + 4a 2)]/2
 FIGURE 1.6 Characteristic Roots and the Unit Circle

Henee: 4 + 4aj + ax2 > ax2 + 4a2 or The stability condition requires that r < \. Thererbre, when plotted on the com-
plex plañe, the two roots a] and a2 must lie within a circle of radius equal to unity.
a2<l+al (1.53)
In the time-series literature it is simply stated that stability requires that all charac-
Thus, the región of stability in Case 1 consists of all points in the región bounded teristic roots lie within the unit circle.
by AOBC. For any point in AQBC, conditions (1.52) and (1.53) hold and d> 0.
In Case 2 (repeated roots), ax2 + Aa2 = 0. The stability condition is \ax\ < 2. Thus, Higher-Order Systems
the región of stability in Case 2 consists of all points on are AOB. In Case 3 (d < 0),
the stability condition is r ~ (-a2) 1/2 < 1. Henee The same method can be used to ñnd the homogeneous solution to higher-order dif-
ference equations. The homogeneous equation for (1.10) is
-a2 < 1 (where a2 < 0) (1.54)
Thus, the región of stability in Case 3 consists of all points in región AOB. For
any point in AOB, (1.54) is satisfied and d< 0.
A succinct way to characterize the stability conditions is to state that the charac - Given the results in Section 4, you should suspect each homogeneous solution to
teristic roots must lie within the unit circle. Consider the semicircle drawn in Figure have the form y¡' = A a? where A is an arbitrary constant. Thus, to find the value(s) of
1.6. Real numbers are measured on the horizontal axis and imaginary numbers are o-, we seek the solution for
measured on the vertical axis. If the characteristic roots ax and a2 are both real, they
can be plotted on the horizontal axis. Stability requires that they lie within a circle of Aa*-fjiiAaf-'1 = 0 (1-56)
radius one. Complex roots will lie somewhere in the complex plañe. If ax > 0, the
roots ax = (a x + iy/d)/2 and a2 = (ax - i\/d)/2 can be represented by the two points or, dividing through by aJ~r\ we seek the valúes of a that solve
2
a n_ axO n-\ _ aiC n- ..._ a/¡ =0 (1.57)
shown in Figure 1.6. For example, ax is drawn by moving axl2 units along the real
axis and Vd/2 units along the imaginary axis. Using the distance formula, the length This «th-order polynomial will yield n solutions for a. Denote these n character-
of the radius r is given by istic roots by o h o2, ..., an. Given the results in Section 4, the linear combination
A X ( \ X * + A-ytW + ... + Ancxnr is also a solution. The arbitrary constants Ax through An can
r = y j(a l /2J 2 + ( d ] / 2
¡/2f ' be eliminated by imposing n initial conditions on the general solution. The a,-may
and, using the fací that i2 = -1, we obtain be real or complex numbers. Stability requires that all real valued a¡ be less than unity
in absolute valué. Complex roots will necessarily come in pairs. Stability requires
r = (-¿7 2 )~l/2 that all roots lie within the unit eircle shown in Figure 1.6.
 PARTICULAR SOLUTIONS FOR DETERMINISTIC PROCESSES 31
30 CHAPTER 1 DIFFERENCE EQUATIONS

Intuition suggests that an unchanging valu é of y (i.e., y t = y t l = ._ = c )

In most circumstances mere is little need to directly calcúlate the characteristic
should solve the equation. Substitute the trial solution y t = c into (1.58) to obtain
roots of higher-order systems. Many of the technical details are included in Appendix
1.2 to this chapter. However, there are some useful rules for checking the stability c = £/Q + ax c + o2c + ... + anc so
conditions in higher-order systems. that
1. In an ;?th-order equation, a necessary condition for all characteristic roots c = ao/(\-al-a2-...-an) (1.59)
to lie inside the unit circle is
As long as (1 - ax - a2 - ... - an) does not equal zero, the valué of c given
by (1.59) is a solution to (1.58). Henee, the particular solution to (1.58) is given
b yy t p = V i 1 - a \ - a i - • • • - a n) -
If 1 - a| - a2 - ... - an = 0, the valué of c in (1.59) is undefined; it is nec-
2. Since the valúes of the a¡ can be positive or negative, a sufficient condition
essary to try some other form for the solution. The key insight is that {yt} is a
for all Characteristic roots to lie inside the unit circle is
unit root process if E^- = 1. Since {yt} is not convergent, it stands to reason that
the constant solution does not work. Instead, recall equations (1.12) and (1.26); ~
¿Kl<i these solutions suggest that a linear time trend can appear in the solution of a
i=l
unit root process. As such, try the solution: yJP — ct. For ct to be a solution it
3. At least one characteristic root equals unity if must be the case that
n
ct = o0 + axc(t - 1) + a2c(í - 2) + ... + anc{t - n) or,
1=1 combining like terms,
Any sequence that contains one or more characteristic roots that equal (1 - ax - a2 - ... - an)ct = a 0 - c(al + 2a2 + 3a3 + ... + non)
unity is called a unit root process. Since 1 - a¡ - a 2 - ... - a n - 0, select the valué of c such that
4. For a third-order equation, the stability conditions can be written as
c = O()/(a\ + 2a 2 + 3a3 + ... + nar¡)
1 — ax — a2 ~ 3 0
a >
For example, let
1 + <3j — ¿72 + ¿?3 > 0
1 - axa3 + a2- a32 > 0 3 + ax + a2 - 3tf3 > 0 or 3 y, = 2 + 0.75^_, + 0.25^,2
- ax + a2 + 3a3 > 0 Here, a{ = 0.75 and a2 = 0.25; {yt} is a unit root process because a{ + a2 =
Given that the fírst three inequalities are satisfied, either of the last 1. The particular solution has the form ct, where c = 2/[0.75 + 2(0.25)} = 1.6. In
two can be checked. One of the last conditions is redundant, given that the the event that the solution ct fails, sequentially try the solutions: yJP — ct2, efi,
other three hold. ... , ctn. For an nth-oráev equation, one of these solutions will always be the
particular solution.
7. PARTICULAR SOLUTIONS FOR
DETERMINISTIC PROCESSES _____________________ CASE 2
Finding the particular solution to a difference equation is often a matter of ingenuity The Exponential Case. Let.^ have the exponential form b(d)rí, where b, d, and
and perseverance. The appropriate technique depends heavily on the form of the {xt} r are constants. Since r has the natural interpretation as a growth rate, we would
process. We begin by considering those processes that contain only deterministic expect to encounter this type of forcing process case in a growth context. We
components. Of course, in econometric analysis, the forcing process wili contain both illustrate the solution procedure using the first-order equation
deterministic and stochastic components.
yt = a0 + axyt_x + bdrt (1.60)
CASE 1 To try to gain an intuitive feel for the form of the solution, notice that if b
= 0, (1.60) is a special case of(1.58). Henee, you should expect a constant to
x t = 0. When all elements of the {xt } process are zero, the difference equa- appear in the particular solution. Moreover, the expression drt grows at .,ae
tion becomes constant rate /*. Thus, you might expect the particular solution to have the form
v, = a0 + a]yt_] + a2yr_2 + ... + any^n (1.58) y¡} = c0 + <r,^r/, where c0 and c¡ are constants. If this equation is actually a
 THE METHOD OF^|jNDETERM1NED COEFFICIENTS 33
32 CHAPTER 1 DIFFERENCE EQUATIONS

Thus, the particular solution will also Contain a linear time trend. You
solution, you should be able to substitute it back into (1.60) and obtain an iden- should have no difficulty foreseeing the solutibn technique \fa] + a2 = i. in tn¡s
tity. Making the appropriate substitutions, we get circumstance—which is applicable to higher órder cases, as well—try multiply-
cQ + cxdrt = a0 + ax[c0 + cxdfi(-1)] + bdrl (1.61) ing the original challenge solution by /. p"
For this solution to work, it is necessary to select c () and cx such that
c 0 = ¿7 0 /(l - ¿7^ and c x = [/?¿/']/(<^' - <?,) 8, THE METHOP QF UISIPETERIV IIÍVIED COEFFlCiENTS
Thus, a particular solution is
At this point, it is appropriate to introduce the first gf two useful methods for finding
p_ a0 bdr , particular solutions when there are stochastic components in the {y(} process. The
key insight of the method of undetermined coefficients is that the pa rticular solu-
The nature of the solution is that jy7 equals the constant ¿70/(l - ax) plus an tion to a linear dsfference equation is necessarily líhear. Moreover, the solution can
expression that grows at the rate r. Note that for \dr\ < 1, the particular solution depend only on time, a constant, and the elements pf the forcing process {xt }. Thus,
converges to aQ/(] - a}). it is often possible to know the exact form of the solution even though the coefficients
If either ax - 1 or ax ~ d*\ use the trick suggested in Case 1. If ax = 1, try of the solution are unknown. The technique involyes positing a solution —called a
the solution c0 ~ ct, and if a] = ¿/r, try the solution cx = ¿6. Use precisely the challenge solution—that is a linear function of all terms thought to appear in the
same methodology in higher-order systems. actual solution. The problem becomes one of finding the set of valúes for those unde-
termined coefficients that solve the difference equation.
The actual technique for finding the coefficients is straightforward. Substitute the
CASE 3 challenge solution into the original difference equation and solve for the valúes of the
undetermined coefficients that yield an identity for all possible valúes of the included
Deterministic Time Trend. In this casé, let the {xr} sequence be represented by variables. If it is not possible to obtain an identity, the form of the challenge solution
the relationship xt = btd where b is a constant and d is a positive integer. Henee is incorrect. Try a new trial solution and repeat the process. In fact, we used the method
n
of undetermined coefficients when positing the challenge solutions yJP - c0 + cxdrí and
■ i=i
yfP - CQ + c x t for Cases 2 and 3 in Section 7. /
To begin, reconsider the simple first-order equation yt = a0 + axy^x + er Since
1a
Since .y, depends on 9 it follows that^_, depends on (/-lK JV-2 depends
you have solved this equation using the iterative method, the equation is useful for
on (t-2)d, and so on. As such, the particular solution has the formyjP = c0 + cxt +
illustrating the method of undetermined coefficients. The nature of the {y,} process is
c2í2 + ... + cdíd. To fínd the valúes of the ch substitute the particular solution
such that the particular solution can depend only on a constant term, time, and the
into (1.62). Then select the valué of each c¡ that results in an identity. Although
individual elements of the {e(} sequence. Since t does not explicitly appear in the
various valúes of ¿/are possible, in economic applications it is com-mon to see
forcing process, t can be in the particular solution only if the characteristic root is
models incorporating a linear time trend (d- 1). For illustrative pur-poses,
unity. Since the goal is to illustrate the method, posit the challenge solution:
consider the second-order equation yt = a0 + aj>'r_j + a^t_2 + bt. Posit the
CO >:
solution yJP = c0 + cxt where c0 and cx are undetermined coefficients.
y^bv+bit + YjXiSt-i (1-64)
Substituting this "challenge solution" into the second-order difTerence equation
yields /=o <
c o + c, / = aQ + a^CQ+c^t- 1)] + ¿7 2[c 0 + c, (/- 2)] + 6r (1.63) where b0, b x , and all the a¿ are the coefficients to be determinad.
Substitute (1.64) into the original difference equation to form
Now select valúes of c0 and Cj so as to forcé equation (1.63) to be an iden-
tity for all possible valúes of t. If we combine all constant terms and all terms b{) + b x t + rvO£> + a,£>_, + a2£t_2 + . . . = a o + a x [ b 0
involving r, the required valúes of c0 and c} are + 6,(/ - 1) + ao£t_x + axe(_2 + . . . ] + e{
C; — b/{\ — O | — <22) C ollectin g lik e t erms , we obt ain
c ( ) = [a 0 ~ (2a 2 + ^i)cj] / (1 - ¿7, - # 2 ) so (bo-ao-a]bo + Í I , Z > ] ) + & I ( 1 -tti)/+ («o ~ 0^
that + (o¡ - ¿7, rv o )f 7 _ 1 + (cv 2 - a x c\\)e t _ 2 + («3 - <i\<*2) e f-3 + • • • =
° (1.65)

c0 = [ao/(\ - a x ~ a2)] ~ [b/(] - a] - ¿72)2](2í:/2 + ü ] )

34 CHAPTER 1 DIFFERENCE EQUATIONS THE METHOD OF UNDETERMINED COEFFICIENTS 35

Equation (1.65) must hold for all valúes of / and all possible valúes ofthe {e,} The form of the solution is "improper" because the sum of the {£f} sequence may
sequence. Thus, each of the following conditions must hold: not be finite. Therefore, it is necessary to impose an initial condition. If the valué yQ
a0 - 1 = 0 Q j is given, it follows that
Oü
-axa0 = 0
Q'2 — ü\Qí\ ~ 0 ^o^o + ZX/
i=0 Imposing the initial condition on the
b o -a o -a x b o + a x b x = 0 bx — improper form of the solution yields (1.26)
oxbx = 0
Notice that the fírst set of conditions can be solved for the a¡ recursively. The /=i
solution of the fírst condition entails setting a 0 = 1. Given this solution for a 0, the
next equation requires ax = ax. Moving down the list, a2 = axax or a2 = ax2. To take a second example, consider the equation
Continuing the recursive process, we flnd o¿¡ — ax'\ Now consider the last two equa- yt = a0 + axyt_x + £t +{3\£t-\ í1-67)
tions. There are two possible cases depending on the valué of ax. \í ax 3* 1, it Again, the solution can depend only on a constant, the elements of the {£,}
immediately follows that bx = 0 and b0 = aQ/(l - ax). For this case, the particular sequence, and / raised to the fírst power. As in the previous example, / does not need
solution is to be included in the challenge solution if the characteristic root differs from unity. To
reinforce this point, use the challenge solution given by (1.64). Substitute this tenta-
tive solution into (1.67) to obtain
b
0+b\t + Yl(*i et~< =«0+ú!1^04- b\ (*-0 + Ylai et-\-A+ et + 01 £t-\
Compare this result to (1.21); you will see that it is precisely the same solution i=0 i=0
found using the iterative method. The general solution is the sum of this particular Matching coeffícients on all terms containing e p £ t _ x , £t-2> - • - ■ » yi e ^ s
solution plus the homogeneous solution Aa\. Henee, the general solution is
ao= 1
ax = ax a0 + (3X [so that a x = a x + j3 x]
l
~l a
1=0 Now, if there is an a 2 = a xa x [so that a 2 = a\(a\ + 0 X )1
a$ = ax a2 [so that a3 = (a x)2(o x + /3X)]
initial condition for>'0, it follows that
a¡ = axa¡_x [so that a¡ = (a l)i-{(a x + (3X)]
l ~~ a \
Matching coeffícients of intercept terms and coeffícients of terms containing /,
/=o
we get
Combining these two equations so as to elimínate the arbitrary constante, we obtain b o = a o + a xb 0- a xb x b x
=a xb x
a °° 00
Again, there are two cases. If ax ^ 1, then bx = 0 and b0 = ao/(\ - a{). The par-
1 "1 /=o [ 1=0 ticular solution is
so that
l
~a^ Í=I
yr = 7~T + Yla\ et-i+a\ \yo-ao/^-a\)\ (1.66)
1 L J The general solution augments the particular solution with the term Aaxl. You are
"1 /=o
left with the exercise of imposing the initial condition for>>0 on the general solution.
It can be easily verified that (1.66) is identical to (1.25). Instead, if o, = 1, bQ can Now consider the case in which ax = 1. The undetermined coefficients are such that
be any arbitrary constant and b] = a0. The improper form of the solution is b\ = a() and b0 is an arbitrary constant. The improper form of the solution is
oc- oo
yt = bo+aot + J2£r-i >', = h + aQt + s f + { \ + /?,) ^ £ t-\
/=0
36 CHAPTER 1 DIFFERENCE EGUATIONS THE METHOD OF UNDETERMINED COEFFICIENTS 37

iO'o is given, it follows that Equation (1.69) must equal zero for all valúes of/. First, consider the case in which
OÜ
í7) -i- ¿¡2 ^± 1. Since (1 — Í/| — C/T) does nol vanish, it is necessai^ to set the valué of b0
1=1
equal to zcro. Given that b2 = 0 and that the coefTicient of / must equal zero, it follows
that b\ must also be set equal to zero. Finally, given that /)¡ = b-> = 0, we must set
Henee, imposing the initial condition, we obtain b{) = ¿/Q/(1 - Í/J - a2). Instead, if Oj + a2 = 1, the solutions for the h¡ depend on the spe-cillc
t-\
valúes of Í/{), Í ? J , and a->. The key point is that the stabllity condition for the homo-geneons
equation is precisely the condition for convei*gence of the particular solution. If anv
i=\
characteristic root ofthe homogeneous equation is equal to imity, apolynomial time trend
will appear in the particular solution. The order ofthe polynomial is the number of
Higher-Order Systems
imitan' characteristic roots. This result generalizes to higher-order equations.
The identical procedure is used for higher-order systems. As an example, let us find If you are really clever, you can combine the discussion ofthe last section with
the particular solution to the second-order equation the method of undetermined coefficients. Find the deterministic portion ofthe partic-
ular solution using the techniques discussed in the last section. Then use the method
y, = a0 + axyt_x + a2y>í_2 + £"/ (1.68) of undetermined coefficients to find the stochastic portion ofthe particular solution.
Since we have a second-order equation, we use the challenge solution In (1.67), for example, set s, = st_\ = 0 and obtain the solution ao/(\ - ¿?j). Now use
y, = b0 + bxt + b2t2 + OQ€É + .«,£>_! + Oi2£t_2 + ... where the method of undetermined coefficients to find the particular solution of yt = axy¡_\
an< tne a are (- ¿7+ f3\£¡_\. Add the deterministic and stochastic components to obtain all compo-
¿> Q , frj, ¿2' ^ 7 ^ie undetermined coefficients. Substituting the
nents ofthe particular solution.
challenge solution into (1.68) yields
[bo +bx t+b2 t 2 ] + ofcér, + a,^, + Qf 2 ^_2 + ... = a0 + a x [b0 + bx (t - 1) + Z? 2 (/ -I) 2 + o^, + A Solved Problem
Q'^^ + c* 2 %-3 + . . . ] + • fl 2 [6 0 + b x {t - 2) + ¿ 2 (/ - 2) 2
+ afc £-, _ 2 + a e
\ í-3 +
<*2 ¿>-4 + - . . ] + 6> To i Ilústrate the methodology using i second-order equation, augment (1.28) with the
There are several necessary and sufficient conditions for the valúes of the cx¡'s to stochastic term £t so that
render the equation above an identity for aü possible realízations of the {ef} sequence: yt = 3 + 0.9^_, - 0.2^_2 + Et ( l - 7 °)
OQ = 1 You have already veritled that the two homogeneous solutions are / íj(0.5) / and
a x — a x a§ [so that cxx = a{\ A->(().4) 1 and that the deterministic portion ofthe particular solution is yf> — 10. To
o¿2 = ax d\ + a2OQ [so that a2 = (¿ÍJ)2 + £721 find the stochastic portion ofthe particular solution, form the challenge solution
a3 = ¿7]Q2 + o2cxx [so that a3 = (ax)3 + 2axa2]

Notice that for any valué of/ > 2, the coefflcients solve the second-order differ-
In contrast to ( 1. 64) , the intercept term b0 is excluded (since we have already
ence equation ctj = axaj_x + a2a.j_2. Since we know OQ and ax, we can solve for all the
Qy iteratively. The properties of the coefficients will be precisely those discussed found the deterministic portion ofthe particular solution) and the time trend b x t is
when considering homogeneous solutions: excluded (since both characteristic roots are less than unity). For this challenge to
work, it must satisfy
1. Convergence necessitates that |¿?2| < 1, ci\ + a2 < 1, and that a2 - al < 1.
Notice that convergence implies that past valúes of the {e{} sequence ulti-
mately have a successively smaller influence on the current valué of yt. - 0.2 [c\ {) s t _ 2 + njf-^ + n^f^ + ajS t __ 5 + . . . ] + £, (1.71)
2. If the coefficients converge, convergence will be direct or oscillatory if Since (1.71) must hold for all possible rcalizations of ¿7, £f_^ st_2, ..., each ofthe
(aj2 + 4a 2) > 0, will follow a sine/cosine pattern if (ax2 + 4a 2 ) < 0, and íollowing conditions must hold:
will "explode" and then converge if (a x2 + 4a 9) = 0. Appropriately setting
n-0 = 1
the ah we are left with the remaining expression:
n-, = 0.9o()
b 2 () ~ a ] ~ a 2 )t 2 + [ b } ( ] - a } ~ a 2 ) + 2b 2 (a } + 2a 2 )]í +
so that o, = 0.9, and for all / > 2,
[bo(\ -al-a2)-¿io^al(b] - b2) + 2a2{b] -2b2)] = 0 (1.69)
i \ i = 0 . 9 ( \ i _ { -0.2n-¡__ 2 (1.72)
38 CHAPTER 1 DIFFERENCE EQUATIONS LAG OPERATORS 39

Now, it is possible to solve (1.72) ileratively so that a 2 = 0.9^ - 0.2cv 0 = 0.61, a, 3. The associative law of multiplication holds for lag operators. We can set
= 0.9(0.61) — 0.2(0.9) = 0.369, and so forth. A more elegant solution method is to L'íJy, = L'(Üy¡) = L'yf i =,v/../_/-. Simiiarly, we can set L'Uy, = Li+Jyt = y¡ .__..
view (1.72) as a second-order difference equation in the {a¡} sequence with initial Note that /,°v, = yt. ' '~J
conditions OQ = 1 and aj = 0.9. The solution to (1.72) is 4. L raised to a negative power is actually a lead operator: L-'yt = yf+i. To
a¡ = 5(0.5)'r- 4(0.4)'" (1.73) explain, definey* = —/ and form Üyt ~ y^¡ — vf+¡-
To obtain (1.73), note that the solution to (1.72) is: a¡=A3(0.5)i + A4(0A)' where A3 5. For |a| < 1, the infinite sum (1 + aL + a 2L2 + a 3íJ + ...)yt = yt /{ 1 - aL).
and A4 are arbitran/ constants. Imposing the conditions OQ = 1 and a-j = 0.9 yields This property of lag operators may not seem intuitive, but it follows
(1.73). If we use (1.73), it follows that: Q 0 = 5(0.5)° - 4(0.4)° = 1; a, = 5(0.5)! - directly from properties 2 and 3 above.
4(0.4)1 = o.9; a 2 = 5(0.5) 2 - 4(0.4) 2 = 0.61; and so on. Proof: Multiply each side by (1 - aL) to form (1 - aL){ \ + aL + a2L2 +
The general solution to (1.70) is the sum of the two homogeneous solutions and a^l? + ...)yt — yv Multiply the two expressions to obtain (1 — aL ■+- aL — a2L-+
the deterministic and stochastic portions of the particular solution: a2L2~£i3¿3 + ...)>>, =yr Given that |a| < 1, the expression anLnyt converges to
oo zero as n —> oo. Thus, the two sides of the equation are equal.
^ = 104 ^(0.5)'4 A 2(0.4)'4 ][>£,_,• (L74) 6. For \a\ > 1, the infinite sum ,1 + (aL)~ l + (aL)- 2 + (aL)~ 3 + . . . ] y t =
i=0 -aLy,/( 1 - aL). Thus,
where the a,- are given by (1.73). oo
Given initial conditions for>> 0 and 3^, it follows that^j and A2 mus.t satisfy y t /(I - aL) = ~(aL)~ ] Y^( üL ^ ^ •
00

yo=lO + Al+A2 + Y,ai£-i (1-75)

Proof. Multiply by (1 - aL) to form (1 - aL)[\ 4- (aL)~ l + (aL)- 2 +
;=o
(aL)-3 + . . . ] y t = —ciLyt. Perform the indicated multiplication to obtain [1 —
00
aL + (aL)~ ] -1 + (aL)~ 2 - (aL)~ ] + (a¿)~ 3 - (oL)~ 2 .. .]y, = -aLyt . Given that
y1 = 10 + A1(0.5; + A2(0.4) + 53ofew (1.76) \a\ > 1, the expression a~nL~nyt converges to zero as n —> oo. Thus, the two
1 =0
sides of the equation are equal.
Although the algebra gets messy, (1.75) and (1.76) can be substituted into (1.74)
Lag operators provide a concise notation for writing difíerence equations. Using
to elimínate the arbitran/ constants:
lag operators, we can write the pth-order equation y¡ — ¿/Q + «j3;/_i + ... + apyt_p 4- et as
y , = 1 0 ^ 0 . 4 )' [ 5 ( y 0 - 1 0) - 1 0 ( y , - 10 )]
(1 - a,L - c;2¿2 - ... _ a pLP)y, = a {) + e, or,
í-2
more compactly, as
+ (0.5)r[10(v1-10)-4(y0-10)] + ^ Qi-£r-/
1=0 ¿(L)yt = «o + EÍ
where A(L) is the polynomial (1 - ¿7jL - a 2 L2 - ... - aJLP)
9, LAG OPERATORS ______________________________ Since A(L) can be viewed as a polynomial in the lag operator, the notation A(\)
is used to denote the sum of the coefficients
If it is not important to know the actual valúes of the coefficients appearing in the par- A{ 1) = 1 - c/j - a 2 ... - a p
ticular solution, it is often more convenient to use lag operators rather than the
method of undetermined coefficients. The lag operator L is defíned to be a linear As a second example, lag operators can be used to express the equation yt = a0 4
operator such that for any valué y{ «iV.-i + ... 4- a p y t _ p 4- e t + f3 x e t _ x + ... + p í{ s t __ q as
A(L)y t = a 0 + B(L)e t
¿Í>',=J' M- Cl-77)
Thus, V preceding.y, simply means to lag yt by / periods. It is useful to remem- where A(L) and B{L) are polynomials of ordersp and q, respectively.
ber the following properties of lag operators: It is straightíbrward to use lag operators to solve linear difference equations.
Again consider the first-order equation y, = a$ 4 a\v,_\ 4 st where |¿?j| < 1. Use the
1. The lag of a constant is a constant: Le = c. definition of L to form
2. The distributive law holds for lag operators. We can set
y í = a0 + a]Ly í + tf (1.78)
(D + U)yt = Üyt + Üyt = vM + y,_j.
 40 CHAPTER 1 DIFFERENCE EQUATIONS SUMMARY 41

Solving for y¡, we obtain In principie, one could use lag operators to actually obtain the coefTicients of the
particular solution. To ¡Ilústrate using the second-order case, considcr yt = (a0 + £■,)/(! -
* = ¥**■ (1-79) axL - a~>L2). If we knew the factors of the quadratic equation were such that (1 — axL -
ciil2) = (I - h xL)(\ ~ b2L), we could write
From property 1, we know tbat La0 = or0, so that ao/(\ - oxL) = ao + axa0 + ax2ao + ...
^ = (a o + 5 / )/[(l-6 1 L)(l -b 2 L)]
=áo/(l -ax). From property 5, we know that £,/(l -¿7]!) = £, + axet_x + O!2^_2 + ....
Combining these two parts of the solution, we obtain the particular solution given If both ¿>j and ¿>9 are less than unity in absolute valué, we can apply property 5
by (1.21). to obtain ^
For practice, we can use lag operators to solve (1.67): yt. ~ oQ + axyt_x + et + j3xel_x, \ao/(l-bi)\ + Y^b\£t-i
where \ax \ < 1. Use property 2 to form (1 - axL)yt = a0 + (1 + (3xL)Et. Solving forj;, yields y i - --------------------------------------------
yt=[a0 + (\+/3xL)e{]/(\-axL) so \-b2 L
that Reapply the rule to ao/(\ - bx) and to each of the elements in the summation EZ?]'
y, = [a 0 / 0 - *i)l + te / (1 - a\L)] + [A¿ M / (1 ~ *i¿)] (1.80) F.t .¿ to obtain the particular solution. If yon want to know the actual coefficients of the
Expanding the last two terms of (1.80) yields the same solution found using the process, it is preferable to use the method of undetennined coefficients. The beauty
method of undetermined coefflcients. of lag operators is that they can be used to denote such particular solutions succinctly.
Now supposej;, = a0 + axyt^x + et but \a x \> 1. The application of property 5 to The general model
(1.79) is inappropriate because it implies that yt is infinite. Instead, expand (1.79) A{L)yt = ao + B{L)et has
using property 6: the particular solution
n OO
y t - a o /A(L) + B(L)^(L)
1 — ai- ,=o
As suggested by (1.82), there is a forward-Iooking solution to any linear difference
equation. This text will not make much use of the forward-Iooking solution since
future realizations of stochastic variables are not directly observable. Some of the
details of forward-Iooking solutions can be found at www.cba.ua.edu/~enders.

10. SUMMARY _______________________________________

Time-series econometrics is concerned with the estimation of difference equations containing
Lag Operators ¡m Higher-Order Systems stochastic components. Originally, time-series models were used for forecasting. Uncovering
the dynamic path of a series improves forecasts because the predictable components of the
We can also use lag operators to transform the >7th -order equation yt = a 0 + a x yf _ x + a2 series can be extrapolated into the future. The growing interest in economic dynamics has
yf~2 + . . . + « „ v,_« + £/ into given a new emphasis to time-series econometrics. Stochastic difference equations arise quite
naturally from dynamic economic models. Appropriately estimated equations can be used for
(1 - a x L - a 2 L 2 - ... - ¿7 / ? L»)V/.'■= a Q + ^ or the interpretaron of economic data and for hypothesis testing.
2 This introductor/ chapter focused on methods of "solving" stochastic difference equations.
yt = (¿20 + £-r)/( 1 - aXL - a2L - ... afTL")
Although iteration can be useful, it is impractical in many circumstances. The solution to a linear
From our previous analysis (also see Appendix 1.2), we know that the stability difTerence equalion can be divided into two parts: a particular solution and a homogeneous solu-
condition is such that the characteristic roots of the equation an - ax a"~] -...-a =0 tion. One complicating factor is that the homogeneous solution is not unique. The general solu-
all lie wiíhin the unit circle. lotice that the valúes of a solving the characteristic equa- tion is a linear combination of the particular solution and atl homogeneous solutions. Imposing n
tion are the reciprocáis of the valúes of L that solve the equation 1 - axL ... - ajj" = initial condiíions on the general solution of an nth-order equation yields a unique solution.
0. In fact, the expression 1 - axL ... - anLn is often called the inverse characteristic The homogeneous portion of a difference equation is a measure of the disequ'úibrium in the
equation, Thus, in the literature, it is often stated that the stability condition is for the initial period(s). The homogeneous equation is especially important in that it yields the charac-
characteristic roots of (1 - axL ... - anLn) to lie outsideot'ihz unit circle. teristic roots; an ?7th-order equation has n such characteristic roots. If all of the characteristic
roots lie within the unit circle, the series will be convergent. As you will see in Chapter 2, there
42 CHAPTER 1 DIFFERENCE EQUATIONS QUESTIONS AND EXERCISES 43

is a direct relationship between the stability conditions and the issue of whe íher an economic b. y, = £/|>V-i + £]( + '^/ (Hint: The form of the solution is
variable is stationary or nonstationary.
The method of undetermined coefficients and the use of lag operators are powerful tools 5. The unit root problem in time-series econometries is concerned with characteristic roots
for obtaining the particular solution. The particular soíution will be a linear function of the cur- that are equal to unity. In order to preview the issue:
rent and past valúes of the forcing process. In addition, this solution may contain an intercept a. Find the homogeneous solution to each of the following (Hint: Each has at least one
unit root}:
temí and a polynomial function of time. Unit roots and characteristic roots o utside of the unit
circle require the imposition of an initial condition for the particular solution to be meaningful. i. y, = 1.5v,_i - 0-5V/-2 + st ii. yt = y,_2 + £,
Some economic models allow for forward-looking solutions; in such circumstances, antici- üí. y, = 2yt_x ~yt_2 + et iv. y t = y t_x + 0.25y t_2 - 0.25^3 + e t
pated fuñare events have consequences for the present period. b. Show that each of the backward-looking solutions is not convergent.
The tools developed in this chapter are aimed at paving the way for the study of time-series c. Show that Equation / can be written entirely in fírst differences; that is, Ay( =
econometries. It is a good idea to work all of the exercises presented below. Characteristic roots, 0.5Av',_| + f}. Find the particular solution for Ay t. (Hint: Find the particular solution
the method of undetermined coefficients, and lag operators will be encountered throughout the for the {Ay t } sequence in tenns of the {st } sequence.)
remainder of the text. d. Similarly transform the other equations into their first-difference form. Find the
backward-looking particular solution, if it exists, for the transformed equations.
QUESTIONS AND EXERCISES ______________________ e. Given an initial condition y 0, find the solution for: y( = a0 —yt_\ + £r
6. A researcher estimated the following relationship for the inflation rate (TT():
1. Consider the difference equation yt = a0 + axyt_x with the initial condition y0. Jill solved 7T( = -0.05 + 0.7 7T,., + 0.6 TT^ + £>
the difference equation by iterating backward a. Suppose that in periods 0 and 1, the inilation rate was 10 percent and 11 percent,
y, = ao + axyt_x respectively. Find the homogeneous, particular, and general solutions for the infla
= a0 + ax(a0 + axyt_2) tion rate.
= ao + aoax + a0a}2 + ... + aoaxf-1 + ax%
b. Discuss the shape of the impulse response function. Given that the United States is
Bill added the homogeneous and particular solutions to obtain yt = ao/(\ - ax) + «^'[yo - not headed for runaway inflation, why do you believe that the researcher's equation
ao/(l-*i)]. is poorly estimated?
a. Show that the two solutions are identical for |¿7j| < 1. 7. Consider the stochastic process y, = a{) + aiyt~2 + £>-
b. Show that for ax = 1, Jill's solution is equivalent to>>, = aQt + y0. How would you a. Find the homogeneous solution and determine the stability condition.
use BilPs method to arrive at this same conclusión in the case that ax — 1? b. Find the particular solution using the method of undetermined coefficients.
2. The cobweb model in Section 5 assumed static price expectations. Consider an alterna- 8. Consider the Cagan (1956) demand for money function in which mt —pt— a — /3(p(+l — p¡).
tive formulation called adaptive expectations. Let the expected price in / (denoted by p¡) a. Show that the backward-looking particular solution forp, is divergent.
be a weighted average of the price in t-\ and the price expectation of the previous
b. Obtain the forward-looking particular solution for/?, in terms of the {mt} sequence.
period. Formally, In forming the general solution, why is it necessary to assume that the money mar -
p] = o¿pt_x 4- (1 - a)p*_, 0<a<1 ket is in long-run equilibrium?
Clearly, when a = 1, the static and adaptive expectations schemes are equivalent. An c. Find the impact multiplier. How does an increase in w /+2 affect/?/? Provide an intu-
interesting feature of this model is that it can be viewed as a difference equation express- itive explanation of the shape of the entire impulse response function.
ing the expected price as a function of its own lagged valué and the forcing variablep,_x. 9. For each of the following, verify that the posited solution satisfles the differenee equa
a. Find the homogeneous solution for p*. tion. The symbols c, c0, and ¿<r0 denote constants.
b. Use lag operators to find the particular solution. Check your answer by substituting
your answer into the original difference equation. (a) y t -y { -\ = 0 yt = c
3. Suppose that the money supply process has the form mt = m + pmt_x + en where m is a (b) v,-^,_i =«0 yt = c + a o t
constant and 0 < p < 1. (c)yt-y t_2 = Q y, = c + c o (r\y
a. Show that it is possible to express mt+n in terms of the known valué mt and the (d) y t - y t -2 = -t v, = c + c o(-\ y + e t + ^_ 2 + e^ + ...
sequence {eí+u ¿r,+2, ...,£>+„}. 10. Part 1: For each of the following, determine whether {yt} represents a stable process.
b. Suppose that all valúes of £t+¡ for / > 0 have a mean valué of zero. Explain how you Determine whether the characteristic roots are real or imaginary and whether the real
could use your result in part a to forecast the money supply n periods into the future. parts are positive or negative.
4. Find the particular solutions for each of the following:
a. v/ = a,v,_1 + ^ + ;V/-i
44 CHAPTER 1 DIFFERENCE EQUATIONS APPENDIX 1.1: IMAGINARY ROOTS AND DE MOIVRE'STHEOREM 45

a. y, - 1.2vM + 0-2>',-2 b' ■>'' ~ l -2-v'-i + °-4^/-2 lf #| = (h. vve can drop subscripts and form
c. v, - 1.2^, - 1.2y,_ 2 d. v, + 1.2v,_i sin(2#) = 2sin(í9)cos(í9)
e. ^; - 0.7v,_i - 0-25v,_2 + 0-175>;/-3 = ° cos(2ft = cos(^)cos(0 - sin(^)sin(<9) (A 1.4)
[/y/M/: (x - 0.5)(JL- + 0.5)(x - 0.7) = x3- 0.7,r2 - 0.25* + 0.175.] The First íask is to demónstrate how to express imaginary numbers in the com-
Part 2: Write each of the above equations using lag operators. Deten-nine the characteris- plex plañe. Consider Figure A l . l in which the horizontal axis measures real numbers
tic roots of the inverse characteristic equation. and the vertical axis measures imaginary numbers. The complex number a + bi can
11. Consider the stochastic difference equation be represented by the point a units from the origin along the horizontal axis and b
units from the origin along the vertical axis. It is convenient to represent the distance
^=0.8^-1 + ^-0.5^!
from the origin by the length of the vector denoted by r. Consider angle #in triangle
a. Suppose that the initial conditions are such that yQ = 0 and €Q = £_x = 0. Now suppose Oab and note that cos((9) = aír and sin(#) = blr. Henee, the lengths a and b can be
that el = 1. Determine the valúes y} through.y5 by forward iteration. measured by
b. Find the homogeneous and particular solutions.
c. Impose the initial conditions in order to obtain the general solution. a = r cos(0) and b = r sin(0
d. Trace out the time path of an e¡ shock on the entire time path of the {yt} sequence. In terms of (Al .2), we can define a = c/j/2 and 6= Vd/2. Thus, the characteris-
12. Use equation (1.5) to determine the restrictions on a and /?necessary to ensure that the tic roots r*j and <>> can be written as:
{yt} process is stable. ÍVJ = a + bi = r[cos(6) + i's'm{0)]
a2 = a - bi = r[cos(#) ~ / sin((9)] (Al.5)
ENDNOTES __________________________________________ The next step is to consider the expressions cv/ and a2 '• Begin with the expres-
¡

1. Another possibility is to obtain the forward-looking solution; such solutions are discussed in Section 10. sion cvj2 and recall that P- = —1:
2. Alternatively, you can substitute (1.26) into (1.17). Note that when et is a puré random disturbance, a,2 - {r[cos(0) + i sin(^)]} {r[cos(<9) + / sin(^)]}
y, — a0 + yt_¡ + et is called a random walk plus drift model.
3. Any linear equation in the variables Xj through xn is homogeneous if it has the form <3)jcj + a2x2 + = r2[eos(6)cos(#) - s¡n(^)sin(^ + 2/ sin(0cos(#)]
... + anxn = 0. To obtain the homogeneous portion of (1.10), simply set the intercept term a0 and
the forcing process xt equal to zero. Henee, the homogeneous equation for (1.10) is yt — a} yt_i + From (A 1.4),
a
2>'i-2 + • • • + a
nyt-n- a,2 = r2[cos(2#) + / sin(26)]
4. lf b > a, the demand and supply curves do not intersect in the positive quadrant. The assumption a
> b guarantees that the equilibrium price is positive. If we continué in this fashion, it is straightforward to demónstrate that
5. For example, if the forcing process is x¡~ £, + (3\€t_\ + fye,^ + • ••» tne impact multiplier is the par- aj' = H[cos(t&) + / sin(/#)] and a^ - rl[cos(l(?) — i sin(/<9)]
tial derivative of yt with respect to et.

APPENDIX 1.1: Imaginary Roots and

de Moivre'sTheorem
Consider a second-order difference equation y¡ = a} yt_\ + ^2>;/-2 Sucn tnat tne ^s~
criminant d is negative [Le., d— a^ + 4a2 < 0]. From Section 6, we know that the full
homogeneous solution can be written in the form
y* = A xa x< + A2a 2< (Al.l)
where the two imaginary characteristic roots are
ai =(a 1 -h/V = ^)/2 and Q 2 = (a, ~ij~^d)¡2 (Al.2)
The purpose of this appendix is to explain how to rewrite and interpret (Al.l) in
terms of standard trigonometric functions. You might fírst want to refresh your mem-
ory concerning two useful trig identities. For any two angles 6\ and ft,
sin(<9j + &¿> = siní^Jcosí^) + cosí^^sinC^)
cos(^ + (h) = cos(^j)cos(6^)- sin(^)sin(6^) (Al.3)
46 CHAPTER 1 DIFFERENCE EQUATIONS APPENDIX 1.2: CHARACTERISTIC ROOTS IN HIGHER-ORDER EQUATIONS 47

Since >>/7 is a real number and ox and a2 are complex, it follows that A¡ and A2 the expression n/ is positive for even valúes of / and negative for odd valúes
of/. Thus, if any of the (\¡ are negative (but less than one in absolute valué),
musí be complex. Although A x and A2 are arbitrary complex numbers, they must have
the solution will tend to exhibit sonie oscillation. If any of the or are greater
the form
than unity in absolute valué, the solution will diverge.
Ax = Bx[cos(B 2) + /sin(£2)] and A2 = 5,[cos(5 2) - / sin(£ 2)] (A 1.7) 2. AH of the cz¿ are real but m < n of the roots are repeated. Let the solution
where Bx and B2 are arbitrary real numbers measured in radians. be such that a.\ = a2 = ... = oim. Cali the single distinct valué of this root a
In order to calcúlate Ax(a xl ), use (Al.6) and (Al.7) to form and let the other n-m roots be denoted by (ym+\ through an. In the case of a
second-order equation with a repeated root, you saw that one solution was
Axax' = Bx[co${B 2) + / sin(£2)]r'[cos(/0 + i sin(/#)] Ai a' and the other was A2íal. With m repeated roots, it is easily verified that
= ^'[cosO^cos^ - sin(5 2 )sin(/<9) + i cos(t0)sin(B 2) + / s\n(t&)cos(B 2 )] ta, /2a', ..., tm~x a- are also solutions to the homogeneous equation. With m
Using (Al.3), we obtain repeated roots, the linear combination of all these solutions is
Axaxí = Bxrí[cos(t0+B2) +ism(t0+B2)] (A 1.8) Axa! +A 2 tó! +Ast 2o! +.. : + A m tm~ Xat + Am+^ m+x +... + A rt a/
You should use the same technique to convince yourself that 3. Some of the roots are complex. Complex roots (which necessarily come
A2a2í = Bxrt[cos(t0+B 2)-is'm(t0+B2)] (A 1.9) in conjúgate pairs) have the form cx¡ db i9. where a.¡ and #are real numbers
and / is defined to be 4—\- For any such pair, a solution to the homoge
Since the homogeneous solution j;/ 7 is the sum of (Al.8) and (Al.9),
neous equation is: Ax{c\\ + iO)f + A?(a¡ — ¡O)1 where Ax and A2 are arbitrary
y/1 = £Ir/ [cos(/0+ B2) + i sin(í#+ B2)] + £1r'[cos(/#+ £ 2) - / sin(í<9 + B2)] constants. Transforming to polar coordinates, the associated two solutions
= 2Bxrf cos(t0+B2) (ALIO) can be written in the form: 0xr 'cos(6!f + /%) with arbitrary constants (3X and
Since Bj is arbitrary, the homogeneous solution can be written in terms of the /^. Here stability hinges on the magnitude of r*\ if \r\ < 1, the system con
arbitrary constants B2 and Z?3 verges. However, even if there is convergence, convergence is not direct
y,h = £3r'cos(/ 0 + B2) (A1.11) because the sine and cosine functions impart oscillatory behavior to the
time path of^. For example, if there are three roots, two of which are
Now imagine a circle with a radius of unity superimposed on Figure A 1.1. The complex, the homogeneous solution has the form
stability condition is for the distance r = Ob to be less than unity. Henee, in the liter-
ature it is said that the stability condition is for the characteristic root(s) to lie w ithin ftr'cos(& +/%) + A^azy
this unit circle. Stability of Higher-Order Systems: In practice, it is diífícult to find the actual valúes
of the characteristic roots. Unless the characteristic equation is easily factored, it is
APPENDIX 1.2: Characteristic Roots in necessary to use numerical methods to obtain the characteristic roots. Fortunately soft-
Higher-Order Equations ware packages such as Mathematica, Maple, or Mathcad can easily obtain the charac-
teristic roots of any specific difference equation. At one time it was popular to use the
The characteristic equation to an /rth-order difference equation is Schur Theorem to determine whether all of the roots lie within the unit circle. Rather
oP-axt\"-x -a2a»-2 . . . - a / 7 = 0 (Al.12) than calcúlate all of these determinants, it is often possible to use the simple rules dis-
As stated in Section 6, the n valúes of a which solve this characteristic equation cussed in Section 6. Those of you familiar with matrix algebra may wish to consult the
are called the characteristic roots. Denote the n solutions by ti], cv 2, ..., an. Given First edition of this text or Samuelson (1941) for the appropriate conditions.
the results in Section 4, the linear combination Axa]/ + A2a2{ + ... + Ananl is also a
solution to (A 1.12)
A prion\ the characteristic roots can take on any valúes. There is no restriction
that they be real versus complex ñor any restriction concerning their sign or magni -
tude. Consider the possibilities:
1. AI1 the cx¡ are real and distinct There are several important subeases.
First suppose that each valué of a,- is less than unity in absolute valué. In
this case, the homogeneous solution (A 1.12) converges since the limit of
each a/ equals zero as t approaches infinity. For a negative valué of ah
 STOCHASTIC DfFFERENCE EQUATION MODELS 49

CHAPTER b rainfall totals for years I through / is a discrete time series. In many economic
applications, / refers to "time" so that /? represents the change in time. However /
need not refer to the type of time interval as measured by a clock or calendar
STATIONARY TIME-SERIES Instead of allowing our measurement units to be minutes, days, quarters, or years
/ can refer to an ordered event number. We could \zt yt denote the outcome of spin
MODELS / on a roulette wheel; j;, can then take on any of the 38 valúes 00, 0, I, ...,36.
A discrete variable y is said to be a random variable (i.e., stochastic) if for any
real number r there exists a probability p(y < r) that y will take on a valué less than
or equal to /-. This defínition is fairly general; in common usage, it is typically implied
that there is at least one valué of r for which 0 < p(y — r) < 1. If there is some r for
which p(y = r) = 1, y is deterministic rather than random.
The theory of linear diíTerence equations can be extended to allow the forcing process It is useful to consider the elements of an observed time series {VQ, y\,y2> • • • » vt)
{xt} to be stochastic. This class of linear stochastic difference equations underlies as being realszations (i.e., outcomes) of a stochastic process. As in Chapter 1, we con-
much of theory of time-series econometrics. Especially important is the Box-Jenkins tinué to let the notation yt to refer to an element of the entire sequence {yt}. In our
(1976) methodology for estimating time-series models of the form roulette example,^, denotes the outcome of spin / on a roulette wheel. If we observe
spins 1 through 7", we can form the sequencey¡, y2, ...,y-por, more compactly, {yt}.
yt = aQ + axyt_x + ... + apyt-p + st + 0\£i~\ + ••• + 0g£,-q In the same way, the term yt could be used to denote gross domestic product (GDP)
Such models are called autoregressive iutegrated moving-average (ARIMA) in time period /. Since we cannot forecast GDP perfectly, yt is a random variable.
time-series models. This chapter has three aims: Once we learn the valué of GDP in period /, yt becomes one of the realized valúes
1. Present the theory of stochastic linear difference equations and consider from a stochastic process. (Of course, measurement error may prevent us from ever
the time-series properties of stationary ARIMA models; a stationary knowing the "true" valué of GDP.)
ARIMA model is called an autoregressive moving-average (ARMA) For discrete variables, the probability distribution of yt is given by a formula (or
model. It is shown that the stability conditions of the previous chapter are table) that specifies each possible realized valué of yt and the probability associated
necessary conditions for stationarity. with that realization. If the realizations are linked across time, there exists the joint
2. Develop the tools used in estimating ARMA models. Especially useful are probability distributionp(y¡ = r^y-i ~ ^2» • • • > yT~ ri) where r¡ is the realized valué of y
the autocorrelation and partial autocorrelation functions. It is shown how in period /. Having observed the first / realizations, we can form the expected valué
the Box-Jenkins methodology relies on these tools to estimate an ARMA of>;/+j,yt+2, • • • » conditioned on the observed valúes of>'j throughyt. This conditional
mean, or expected valué, ofy(+¡ is denoted by Ef[yt+i \ y(,yt_\, ..., y\] or Etyt+i.
model from sample data.
Of course, if yt refers to the outcome of spinning a fair roulette wheel, the prob-
3. Consider various test statistics to check for model adequacy. Several exam- ability distribution is easily characterized. In contrast, we may never be able to cora-
ples of estimated ARMA models are analyzed in detail. It is shown how a pletely describe the probability distribution for GDP. Nevertheless, the task of
properly estimated model can be used for forecasting. economic theorists is to develop models that capture the essence of the true data-
generating process. Stochastic difference equations are one convenient way of mod-
1. STOCHASTIC DIFFERENCE EQUATION MODELS eling dynamic economic processes. To take a simple example, suppose that the
Federal Reserve's money supply target grows 3 percent each year. Henee,
In this chapter, we continué to work with discrete, rather than continuous, time-
series models, Recall from the discussion in Chapter 1 that we can evalúate the func- int = 1.03m*__j (2.1)
tion v =/(/) at /0 ar*d /0+ h to form so that, given the initial condition m*, the particular solution is
Av=./l'o + /?)-/l¿'o) mt = (1.03) m0
As a practical matter, most economic time-series data are collected for discrete • where:m * = the logarithm of the money supply target in year /
time periods. Thus, we consider only the equidistant intervals / 0. ÍQ+H, to+2h, /0+ Wo — the initial condition for the target money supply in period zero Of course,
3/7, ..., and conveniently set h — 1. Be careful to recognize. however, that a discrete the actual money supply (ni t ) and the target need not be equal. Suppose that at
time-series implies that /, but not necessarily v,. is discrete. For example, although the end of period / — 1, there exists tnt_\ outstanding dollars that are carried forward
Scotland's annual rainfall is a continuous variable, the sequence of such annual into period t. Henee, at the beginning of t there are tnt_\ dollars so that
48
50 CHAPTER 2 STATIONARYTIME-SERIES MODELS ARMA MODELS 51

the gap between the target and the actual money supply is /??* — mt_\. Suppose that the a moving average of order q and is denoted by MA(cy). To illustrate a typical moving
Fed cannot perfectly control the money supply but attempts to change the money sup- average process, suppose you vvin SI if a fair coin shows a head and lose $l if it shows
ply by p percent (p < 100%) of any gap between the desired and actual money sup- a tail. Denote the outcome on toss / by s, (i.e., for toss /, e, is either+$l or-$l). ffyou
ply. We can model this behavior as want to keep track of your hot streaks, you might want to calcúlate your average win-
Amt = p[m* - m,_, ] + £t nings on the last four tosses. For each coin toss /, your average payofTon the last four
tosses is 1/4,5, + \/4st_{ + l/4t,_2 + l/4£*/_3. In terms of (2.3), this sequence is a moving
or using (2.1), we obtain average process such that /3¡ = 0.25 for / < 3 and zero otherwise.
m, = p{\ .03)' m.Q + (1 -P)mr_j + et (2.2) Although the {£",} sequence is a white-noise process, the constructed {x¡} sequence
will not be a white-noise process if two or more of the (3¡ differ from zero. To illustrate
where et is the uncontrollable portion of the money supply.
using an MA(l) process^ set fl0 = 1, f3x = 0.5a and all other /3¡ = 0. ín this circumstance,
We assume the mean of et is zero in all time periods. E(x¡) = E(et + 0.5£>_i) = 0 and varCr,) = var(^ + 0.5e>_,) = 1.25o2. You can easily con-
Although the economic theory is overly simple, the model does illustrate the key vince yourself that E{x() = E(xt_s) and that var^v,) = var^^.) for all s. Henee, the first
points discussed above. Note the following: two conditions for {.v,} to be a white-noise process are satisfied. However E{xtxt_{) —
-r 1. Although the money supply is a continuous variable, (2.2) is a discrete dif- E[(q + 0.5£>_1)(£>_, + 0.5£>_2)] = E(£tet^ + 0.5(€t^)2 + 0.5^^2 + 0.25^,£>_2) = 0.5o2.
ference equation.; Since the forcing process {£>} is stochastic, the money sup- Given that there exists a valué of s ^ 0 such that E(XfXt_s) ^ 0, the {x(} sequence is not
ply is stochastic; we can cali (2.2) a linear stochastic difference equation. a white-noise process.
2. If we knew the distribution of {£,}, we could calcúlate the distribution for Exercise 1 at the end of this chapter asks you to fmd the mean, variance, and
each element in the {mt} sequence. Since (2.2) shows how the realizations covariaríce, of your hot streaks in coin tossing. For practice, you should work that
of the {mt} sequence are linked across time, we would be able to calcúlate exercise before continuing.
the various joint probabilities. Notice that the distribution of the money
supply sequence is completely determined by the parameters of the differ 2. ARMA MODELS ___________________________________
ence equation (2.2) and the distribution of the {st} sequence.
It is possible to combine a moving average process with a linear difference equation
3. Having observed the fírst t observations in the {mt} sequence, we can to obtain an autoregressive moving average model. Consider the p-th order differ-
make forecasts ofm/+1, mr+2, .... For example, updating (2.2) by one ence equation:
period and taking the conditional expectation, the forecast ofm, +] is: P
Et mm = p{\my^ml+ (1 -p)mt. yt = a0 + Yjli >'/-/ + xt (2-4)
• Before we proceed too far along these lines, let's go back to the basic building /=l
block of discrete stochastic time-series models: the white-noise process. A sequence Now let {xt} be the MA(f/) process given by (2.3) so that we can write
{et} is a white-noise process if each valué in the sequence has a mean of zero, a con- p q
stant variance, and is uncorrelated with all other realizations. Formally, if the notation yt = ao + Yl a¡yt-i +Yl(3i£t-í (2-5)
E(x) denotes the theoretical mean valué of x, the sequence {st} is a white-noise
process if for each time period / We follow the convention of normalizing units so that 0Q is always equal to
£0-,) = £0r M) = . . . = 0 unity. If the characteristic roots of (2.5) are all in the unit circle, {yt} is called an
E(e?) = £(£?_,) = ... = o2 [or var(^) = var(£r M) = ... = c¿] autoregressive moving-average (ARMA) model for>',. The autoregressive part of
E{st £t_s) = E(eH eH_s) = 0 for all y and s [or cov(¿rr, £{_s) = cov^-, eH_s) = 0] the model is the difference equation given by the homogeneous portion of (2.4) and
In the remainder of this text, {et} will always refer to a white-noise process and the moving average part is the {x¡} sequence. II the homogeneous part of the differ-
cP- will refer to the variance of that process. When it is necessary to refer to two or ence equation contains p lags and the model for xt contains q lags, the model is
more white-noise processes, symbols such as {e\{} and {¿¿/I w*^ ^e used- Now, use a called an ARMA(p, q) model. If q = 0, the process is called a puré autoregressive
white-noise process to construct the more interesting time-series process denoted by AR(p) and ifp = 0, the process is a puré moving-average process
Q
denoted by MA(^). In an ARMA model, it is perfectly permissible to allowp and/or
xt = Yfíi£t-i ' (2.3) <? to be infinite. In this chapter we consider only models in which all of the charac-
Í=0 teristic roots of (2.5) are within the unit circle. However, if one or more characteristic
For each period /, xt is constructed by taking the valúes er £,_], ..., et_<i and mul- roots of (2.5) is greater than or equal to unity, the {yt} sequence is said to be an
tiplying each by the associated valué of 3¡. A sequence formed in this manner is called integrated process and (2.5) is called an autoregressive integrated moving average
(ARIMA) model.
52 CHAPTER 2 STATIONARYTIME-SERIES MODELS

Treating (2.5) as a difíerence equation suggests that we can "solve" for yf in terms
of the {e{} sequence. The solution of an ARMA(/;, q) model expressingj;, in terms of
the {e}} sequence is the moving-average representaron of yv The procedure is no
different from that discussed in Chapter 1. For the AR(1) model yt = aQ + axyt_x + £,, the
moving-average representation was shown to be
oo
y, = a 0 I (1 - a x ) + XVl "'-' i
=0
For the general ARMA(p, g) model, rewrite (2.5) using lag operators so that

i=l 1=0

so that the particular solution for j;, is

yt=Uo+Í:^t-i / I-¿>L' . (2.6)
I , «=o JI /=i
Fortunately, it will not be necessary for us to expand (2.6) to obtain the specific periods. If you knew that the output was stationary, you could approximate the mean
coefficient for each element in {et}. The important point to recognize is that the level of output by
expansión will yield an MA(oo) process. The issue is whether such an expansión is 20
convergent so that the stochastic difference equation given by (2.6) is stable. As yon
will see in the next section, the stability condition is that the roots of the polynomial
(l - £a¿Z/) must lie outside of the unit circle. It is also shown ifyt is a linear stochas- In using this approximation, you vvould be assuming that the mean was the same
Í tic difference equation, the stability condition is a necessary condition for the time-
series {yt} to be stationary.
for each period. Formally, a stochastic process having a finite mean and variance is
covariance stationary if for all / and / — 5,
E(y() = E(yt_ s) = // (2.7)
3- STATIONAR1TY _________________________________
Suppose that the quality control división of a manufacturing firm samples four £[&, ~ /O2] = E[(yt-S ~ /¿)2J = °] tvar(>V) = var0V_,) = a]] (2.8)
machines each hour. Every hour, quality control fínds the mean of the machines' out-
^[0 ;/ ~ /0O/-.v- /<)] = ^tO'/_7 - /*)ívH_ s ~ //)] = %
put levéis. The plot of each machine's hourly output is shown in Figure 2.1. If yit rep-
resents machine yfs output at hour /, the means (yt) are readily calculated as [covO-V, yt-s) = co\-(yH, yH_ s) = y,] (2.9)
wherc //, o\, and 7? are all constants.
In (2.9), aílowing s = 0 means that 70 ls equivalent to the variance of yf. Simply
í=l For hours 5, 10, and 15, these mean put, a time-series is covariance stationary if its mean and all autocovariances are
valúes are 4.61, 5.14, and 5.03, respectively. unaflected by a change of time origin. In the literature, a covariance stationary
The sample variance for each hour can similarly be constructed. Unfortunately, '' process is also referred to as a weakly stationary, second-order stationary, or wide-
applied econometricians do not usually have the luxury of being able to obtain an sense stationary process. (Note that a strongly stationary process need not have a
ensemble (i.e., múltiple time-series data of the same process over the same time finite mean and/or variance). The text considers only covariance stationary series so
period). Typically, we observe only one set of realizations for any particular series. that there is no ambiguity in using the terms stationary and covariance stationary
Fortunately, if {yt} is a stationary series, the mean, variance, and autocorrelations interchangeably. One further word about terminology: In multivariate models, the
can usually be well approximated by sufílciently long time averages based on the term autocovariance is reserved for the covariance between yt and its own lags. Cross-
single set of realizations. Suppose you observed only the output of machine 1 for 20 covariance refers to the covariance between one series and another. In univari-ate
time-series models, there is no ambiguity and the terms autocovariance and
covariance are used interchangeably.
 54 CHAPTER 2 STATIONARYTiME-SERIES MODELS STATIONARITY 55

For a covariance stationary series, we can define the autocorrelation between>', which is also fínite and time-independent. Finally, it is easily demonstrated that the
and yt_s as limiting valúes of all autocovariances are finite and time-independent:
Ps = %/7o E[{yt - fii)(yt_s - //.)] = E{[€t + ax£,_, + (a,)2et_2 + . . . ] •
- -" [St_s + ^,^_,_1 + (^,)2tV,-2 + ••■]}
where j0 and ys are defmed by (2.9).
= ^(a^Yll + (a { ) 2 + (^i) 4 + . . . ] = ^|M1 - (*l) 2 ] (2-14)
Since % and 70 are time-independent, the autocorrelation coefficients ps are also
time-independent. Although the autocorrelation between yt and yt_x can differ from In summary, if we can use the limiting valué of (2.10), the {yt} sequence will be
the autocorrelation between yf and v,_2, the autocorrelation between yt and yt_x must stationary. For any given>?0 and | a, | < 1, it follows that / must be sufficiently large.
be identical to that between yt_s and j^^j. Obviously, p0 = 1. Thus, if a sample is generated by a process that has recently begun, the realizations
may not be stationary. It is for this very reason that many econometricians assume
Stationarity Restrictions for an AR(1) Process that the data-generating process has been occurring for an infinitely long time. In
practice, the researcher must be wary of any data generated from a "new" process.
For expositional convenience, first consider the necessary and sufficient conditions For example, {j;,} could represent the daily change in the dollar/mark exchange rate
for an AR(1) process to be stationary. Let beginning immediately after the demise of the Bretton Woods fixed exchange rate
systcm. Such a series may not be stationary due to the fact there were detenninistic
y t = a 0 + ax y t _ x + e t
initial conditions (exchange rate changes were essentially zero in the Bretton Woods
where et = white-noise. era). The careful researcher wishing to use stationary series might consider excluding
—k> Suppose that the process started in period zero, so that^Q is a deterministic ini- some of these earlier observations from the period of analysis.
tial condition. In Section 3 of the last chapter, it was shown that the solution to this Little would change were we not given the initial condition. Without the initial
equation is (see also Question 2 at the end of this chapter) valué ^Q, the sum of the homogeneous and particular solutions for^ is
co
y/ = a0'(l-«l) + DaU-«+>U«l)' (2.15)
i=0 i=0 /=0
Taking the expected valué of (2.10), we obtain where A = an arbitrary constant. *.
/-i If you take the expectation of (2.15), it is clear that the {yt} sequence cannot be
stationary unless the expression A(ax)1 is equal to zero. Either the sequence must have
i=0 started infinitely long ago (so that a¡r — 0) or the arbitrary constant A must be zero.
Updating by s periods yields Recaí 1 that the arbitrary constant is interpreted as a deviation from long-run equilib-
rium. The stability conditions can be stated succinctly:
f+JT- l
1. The homogeneous solution must be zero' Either the sequence must have
/=0 started infinitely far in the past or the process must always be in equilib-
Comparing (2.11) and (2.12), it is clear that both means are time dependent. rium (so that the arbitrary constant ¡s zero).
Since Eyt is not equal to EyJ+s, the sequence cannot be stationary. However, if / is 2. The characteristic root ax must be less than unity in absolute valué.
large, we can consider the limiting valué of yt in (2.10). If | ax | < 1, the expression These two conditions readily generalize to all ARMA(/?, q) processes. We know
(tfjXvo converges to zero as t becomes infinitely large and the sum ao[l + ax + (ox)2 that the homogeneous solution to (2.5) has the form
■+■ (ax)3 + ...] converges to ao/(l - a{). Thus, as t —♦ oo and if | ax | < 1
¿A«í
1 fl
1 =1
~ l í=0
Now take expectations of (2.13) so that for sufficiently large valúes of t, Ey, = or i f there are m repeated roots,
ao/(l - oj). Thus, the mean valué of y¡ is fínite and time independent so that Eyt =
Eyt_s ~ ao/(l - ax) = /J íov all í. Turning to the variance, we find «¿v'+ E A«Í
1=1 i=m + l
£Cv, - M)2 = £[(*, + fl,£-M + {arfet_n + ...)2] where the A¡ are all arbitrary constants, a is the repeated root, and the a¡ are the dis-
= o2[l +(^1)2 + ( a 1 ) 4 + . . . ] = ^/(l -(^i)2)
tinct roots.
56 CHAPTER 2 STATIONARYTIME-SERIES MODELS " STATIONARY RESTRICTIONS FOR AN ARMA(R Q) MODEL 57

If any portion of íhe homogeneous equalion is present, Ihe mean, variance, and
all covariances will be time-dependent. Henee, for any ARMA(p, q) model, station-
arity necessitates that the homogeneous solution be zero. The next section addresses COEFFICIENTS OF THE ARMA(2,1) PROCESS: Vt
the stationarily restrictions for the particular solution. = Lóy^j - 0.9yt_2 + et + 0.5st_j
If we use the method of undetermined coefficients, the a¡ must satisfy:
4. STATIONARY RESTRICTIONS FOR AN
a0 = 1
ARMA(p, q) MODEL _______________________________ ai = 1.6 + 0.5 henee, o:] = 2.1
cx¡ = \.6a¡_x ~ 0.9av_2 for all / = 2, 3, 4, ...
As a prelude to the stationarity conditions for the general ARMA(p, q) model, first
consider the restrictions necessary to ensure that an ARMA(2, 1) model is stationary. Notice that the coefficients follow a second-order ditTerence equation with imaginar/ roots.
Since the magnitude of the intercept temí does not aflect the stability (or stationarity) lf we use de Moivre's Theorem, the coefficients will satisfy a,- = 0.949'' /?1cos(0.567/ + /5t>)
conditions, set aQ = 0 and write
Imposing the initial conditions for OQ and a\ yields
y¡ = a \yt-\ + a2 );Í-2 + et + P\ et-\ (2-] 6) 1 = /?jcos (/%) and 2.1 - 0.949/9, cos(0.567 + /^)
From the previous section, we know that the homogeneous solution must be Since /?i= l/cos(/%), we seek the solution to
zero. As such, it is only necessary to fínd the particular solution. Using the method of cos(/%) - (0.949/2.1) • cos(0.567 + /%) = 0
undeteirnined coefficients, we can write the challenge solution as
oo From a trig table, the solution for Sj is -1.197. Henee, the a¡ satisfy (-
1/1.197) • 0.949 1' • cos(0.567i - 1.197)
Altematively, we can use tlie initial valúes of OQ and ai to find the other a¡ by iteration. The
For (2.17) to be a solution of (2.16), the various a¡ must satisfy O QE, + <x\£t-\ + sequence of the a¡ is shown in the graph below.

^2£ t_ 2 + c*j£>_ 3 + ... = a^OQS^ + a x £ t_ 2 + a' 2¿f,_3 + Q^£¡ f ^ 4+ ...)

+ a2(ao£f-2 + <*1 ei-3 + a2st-4 + Q3^/-5 + • • ■) + et + P\ et~\
To match coeffícients on the terms containing e n £t _x, et _2, ..., it is necessary to
set
1. GO =1
2. aj = a] OQ + /?] => a x = a x + 0 X
3. a¡ = aj a¡__\ + a2a¡_2 for all i > 2
The key point is that for / > 2, the coefficients satisfy the difference equation
a¡ - a] a,.] + a 2 a/_o. lf the characteristic roots of (2.16) are within the unit circle, The first 10 valúes of the sequence are
the {cty} must constitute a convergent sequence. For example, reconsider the case /0 1 2 3 4 5 6 7 8 9 10
in which ax = 1.6, a0 = -0.9, and let flx = 0.5. Worksheet 2.1 shows that the coefficients V 1.00 2.10 2.46 2.046 1.06 -0.146 1.187 -1.786 -1.761 -1.226 -0.378
satisfying (2.1*7) are 1, 2.1, 2.46, 2.046, 1.06,-0.146, .... (also see Worksheet 1.2 of
the previous chapter).
Henee, var(y,) = var0 ; r _j) ^ o r a ^ t an<
^ S - Finally, the covariance between y t and;^ is
To verify that the {y,} sequence generated by (2.17) is stationary, take the expec-
tation of (2.17) to form Evf = £>>,_,- = 0 for all / and i. Henee, the mean is finite and CovO';, v,_,) = E[(£ t + ajt^j + Oí 2 e t _ 2 + .-0(^-1 + <*\£ t -2 + Q
2w-3 + a
3 s t-4 + ■ • • ) ] =
time-invariant. Since the {£,} sequence is assumed to be a white-noise process, the 2
cr (a^ + t*2 \ a + a a
3 '2 +
• • •)
variance of yf is constant and time-independent; that is, C ov^, y t _ 2 ) = £[ (£ -, + a x s t _ v + ^o -í -2 * ••■)(^ -2 + a
l"/ -3 +
°'2 w-4 + a
'3 ^-5 +
•••)]
Var(v,) = £[(a<)£, + ^l^-l + <*2€t-2 + °3-/-3 + •••)21 = c¿(c\ 2 + o 3 A| + a 4 ci2 + • • • )
oo so that
= <7~ l^af i=0
CovO',, .V/-.y) = ^(o 5 + cv vH a-j + a s+2 a 2 + ...) (2.1 8)
■
58 CHAPTER 2 STATIONARY TIME-SERIES MODELS
STATIONARY RESTRICTIONS FOR AN ARMA(P, Q) MODEL 59

Thus, cov(y,, yf-s) is constant and independent of t. Conversely, if the character- only depends on the number of periods separating the variables (i.e., the valu é of s)
istic roots of (2.16) do not lie within the unit circle, the {a¡} sequence will not be con- but not on the time subscript /.
vergent. As such, the {yt} sequence cannot be convergent.
In summary, the necessary and sufTicient conditions for any MA process to be sta-
It is not too difficult to generalize these results to the entire class of ARMA(/?, q)
tionary are for the sums (1), £(#)2 and of (2), (/% + fi{f3s+] + í^/%+2 + • • • ) to be finite.
models. Begin by considering the conditions ensuring the stationarity of a pur é
Since (2) must hold for all valúes ofs and /30 = 1, condition (1) is redundant. The direct
MA(oo) process. By appropriately restricting the ¡3h all of the finite-order MA(q)
implication is that a finite-order MA process will always be stationary. For an infinite-
processes can be obtained as special cases. Consider
order process, (2) must hold for all .y > 0. Some of the details involved with máximum
oo
likelihood estimation of MA processes are discussed in Appendix 2.1 of this chapter.
i=0
Stationarity Restrictions for the
where {et} = a white-noise process with variance o2. Autoregressive Coefficients
We have already determined that {xt} is not a white-noise process; now the issue
Now consider the puré autoregressive model
is whether {x{} is covariance stationary. Given conditions (2.7), (2.8), and (2.9), we
P
ask the foliowing:
yt = ao + Y2aiyt-i + £t (2.19)
1. Is the mean finite and time-independent? Take the expected valué of xt and
remember that the expectation of a sum is the sum of the individual expec - If the characteristic roots of the homogeneous equation of (2.19) all lie inside the
tations. Therefore, unit circle, it is possible to write the particular solution as
E(x,) = E{e t + (3 x £ t _ x + fce^j, + ...)
= Eet + 0 xEet_ x + foEs^ + ... = 0 V, = —~- + & £t-i (2-20)
Repeat the procedure with x,_ s
i=\
E{x,_s) = £(£,_, + /?, £-,_,_, + /%e,_*-2 + . . . ) = 0 Henee, all elements where the a¿ — undetermined coefficients.
in the {xf } sequence have the same fin i te mean (// = 0). Although it is possible to find the undetermined coefficients {a?,-}, we know that
2. Is the variance finite and time-independent? Form var(x,) as (2.20) is a convergent sequence so long as.the characteristic roots of (2.19) are inside
Varí*,) = E[{e t + f3 x e t _, + fre^ + ...) 2 ] the unit circle. To sketch the proof, the method of undetermined coefficients allows
us to write the particular solution in the form of (2.20). We also know that the
Square the term in parentheses and take expectations. Since {st} is a white- sequence {a¡} will eventually solve the difference equation
noise process, all terms E£t £t _s = 0 for s ^ 0. Henee
or, - - ¿7, a ¡_ { -a 2 a i _ 2 -. .. - a p a;_ p = 0 (2.21)
Var(x,) = E(et)2 + (/?,)2£(£:M)2 + (fr)2E(£}_2)2 + • ■ • = If the characteristic roots of (2.21) are all inside the unit circle, the {a¡} sequence
^[ l + ( ^i ) 2 + ( ^) 2 + . . . ] will be convergent. Although (2.20) ís an infinite-order moving average process, the
As long as E(#)2 is finite, it follows that var(x,) is finite. Thus, £(/?7)2 being convergence of the MA coefficients implies that Ea, r is finite. Thus, we can use
finite is a necessary condition for {xr} to be stationary. To determine whether (2.20) to check the three conditions for stationarity. Since cv 0 = 1,
var(jc;) = var(xr_5), form Eyt = Eyt_s = ao/( 1 - T.a¡)
Var(x(_s) = E[{ E{_S + Í3x£t_s_, + /%^_ 2 + --.) 2 ] = ^[1 + (A) 2 + W+ • • • ] You should recall from Chapter 1 that a necessary condition of all characteristic |
Thus, var(jcr) = var(x,_5) for all r and t-s. roots to lie inside the unit circle is 1 - J2a¡ > 0. Henee, the mean of the sequence is
3. Are all autocovariances finite and time-independent? First form finite and time-invariant.
E(xtx,_s) as Var( yt) = E[(et + rv, cf _., + ot2£t-2 + a'3-/-3 +'• • -)2 ] = crT,c^ and
Va i t>7-¿)
E[x jXí, s] = E[(s r + (3 x et_ x + ^£>. 2 + •••X^-s + A^-.-i + /%^-,-2 + ••■)] = £r(c,_5+^/-,v~l + a '2"/-A-I +«2f/-5-2 + a 5 £ "f-.v-3+---) 2 ]= < j2 ¿ a á 2
Carrying out the multiplicaron and noting that E(£ t£ f _ s ) = 0 for s ^ 0, we get Given that En¡ is finite, the variance is finite and time-independent.
E(x¿c,_s) = <¿<J35 + /?, 4+, + #>4+2 + ...) Cov0V, y,_s) = E[(£, + a,£,_, + a'2c,_2 + • • • )(-V.v + al¿>-.v-l + ^2-/-,v-2 + • • • ) ] =
(¿(as + a, ns+l + n2as+2 + • • •)
Restricting the sum j3s + /^i/^+i + 02$s+2 + ■ • • lo be fmite means that E{xtXj_s) is
" ■
finite. Given this second restriction, it is clear that the covariance between .Y, and xt_ s
I
60 CHAPTER 2 STAT1ONARY TIME-SERIES MODELS

Thus, Ihe covariance between y¡ and yr_s is constá'nt and time-invariant for all /
and í—s. Nothing of substance is changed by combining the AR(p) and MA(g) mod-
els into the general ARMA(p, q) model:
p

/=!

If the roots of the inverse characteristic equation lie outside of the unit circle
[i.e., if the roots of the homogeneous form of (2.22) lie inside the unit circle] and if
the {xt} sequence is stationary, the {y,} sequence will be stationary. Consider
,, = _fo_ + —£ ----- + -ÉH±¿- + Jlf^u. + ... (2.23)
1
P P P P

With very little effort, you can convince yourself that the {yt} sequence satisfíes
the three conditions for stationarity. Each of the expressions on the right-hand side of
(2.23) is stationary as long as the roots of 1 — HafJ are outside the unit circle. Given
that {xt} is stationary, only the roots of the autoregressive portion of (2.22) determine
whether the {yr} sequence is stationary.

5.THE AUTOCORRELATION FUNCTION ______________

The autocovariances and autocorrelations of the type found in (2.18) serve as useful
tools in the Box-Jenkins (1976) approach to identifying and estimating time-series mod-
els. We illustrate by considering four important examples: the AR(1), AR(2), MA(1), and
ARMA(Í, 1) models. For the AR(1) model, yt = aQ + a^y^ + et, (2.14) shows

7 5=^( £2 1 >V[l-(a1) 2]
Fonning the autocorrelations by dividing each ys by 70, we fmd that/^ = \,p] = ox,
Pl = (a\)2* •'••> Ps = (a\)s- P°r an AR(1) process, a necessary condition for stationarity is
for I a] \ < 1. Thus, the plot of ps against s—called the autocorrelation function
(ACF) or correlogram—should converge to zero geometrically if the series is station-
ary. If ai is positive, convergence will be direct, and if a^ is negative, the autocorrela-
tions will follow a dampened oscillatory path around zero. The first two graphs on the
left-handside of Figure 2.2 show the theoretical autocorrelation functions for a^ = 0.7 process by use of the method of undetemiined coeiTicients. Now we want to illustrate an
and ¿7] = -0.7 respectively. Here, pQ is not shown since its valué is necessarily unity. alternative technique using the Yule-Walker equations. Multiply the second-order dif-
ference equation by y]_s for s = 0, s = 1, s = 2, ... and take expectations to form
The Autocorrelation Function of an AR(2) Process
Eyt yt-\ - «■j l £)í r-i.v /-i + a
iEyi-zyt-\ +
^c"/-v /-i
Now consider the more complicated AR(2) process vr = tf]V,-i + a2.Y't-2 + Ev We omn an Eyiy,-2= o'\Eyi-\.vi-2+ a2^y ¡-2X1-2+ EztXt-2
intercept teirn (a{)) since it has no effect on the ACF. For the second-order process to be
stat¡onary,-\ve know that it is necessary to restrict the roots of (1 - a^L - a2L2) to be out-
side the unit circle. In Section 4. we derived the autocovariances of an ARMA(2, 1) Ey^'t-s = : fl |í)'/-i. li /-.v + a 2 E yt-2Xt-s + Es
iyts . (2.24)
62 CHAPTER 2 STATIONARY TIME-SERIES MODELS ^^^B^^B^
THE AUTOCORRELATION FUNCTION 63
; al E ;
By defínition, the autocovariances of a stationary series jS^Hp^ y/) í-s ~
The Autocorrelation Function of an MA(1) Process
Eyt_syt — Eyt_kyt_k_s = js. We also know that Eety t = o2 and;^Hp;'O. Henee, we
can use the equations in (2.24) to forrn IÍ H HB K' Next consider the MA(I) process yt = et + 0£t-\- ^ga^n' we can obtain the
70 = a\ Ti + *272 + v1 i|JH^ (2'25) Yule—Walker equations by multiplying yt by each yt_s and take expectations
71 = <*\ 70 + *27l IÍSBV (2 26)
' 70 = varCy,) = Ey(yt = E[{et + /?£,_,)(£, + tfs^)] = (I + P2)a2
2
71 = Eytyt_x = E[(st + pe(_x ){et_x + (3et_2)] = fia
2 27
7s = "i T,-l + *2%-2 IB- (- >
and
Dividing (2.26) and (2.27) by 70 yields ^Í I ÍBBK
= Eytyt_s = E[(£f + /3e(_x)(e(__s + f3et_s_^)] = 0 for all s > i
p x = a x p 0 + ar^ ifluV ■ (2-28) 7y

Henee, dividing each -ys by 70, it can be immediately seen that the ACF is sim-
^ = tf i/V-i + *2¿V-2 9^B (2'29)
ply p0 - I, pj = 0/(\ + /52), and ps = 0 for all 5 > I. The third graph on the left-hand
We know that p$ - 1, so that frorn (2.28), px = ¿7j/(l - a2). ^S^^K? can ^nd aM /^ for side of Figure 2.2 shows the ACF for the MA(l) process ^, = et — Q.let__x. As an exer-
s > 2 by solving the diñerence equation (2.29). For examPV¡ÍlBft2> and s = 3, cise, you should demónstrate that the ACF for an MA(2) process has two spikes and
/>2 = (^1)2/(l-^2) + ^2 S»: then cuts to zero.
y>3 = *i[(*i) 2/(i -^ 2 ) + ^2] +
^^(^«■p
Although the valúes of the ps are cumbersome to derive, ^flBBg&Hy character- The Autocorrelation Function of an ARMA(1, 1) Process
ize their properties. Given the solutions for p§ and pi, the key'j^f^^Bnote is that the Finally, let y( = axyt_x + et + P\St_x. Using the now-familiar procedure, we fínd the
ps all satisfy the diflerence equation (2.29). As in the g enera^,S^BEa second-order
Yule-Walker equations
difFerence equation, the solution may be oscillatory or direct.^iBHBilí* tne stationar-
ity condition for yt necessitates that the characteristic roots °J|iBSHpG inside of the Eytyt = axEyt_xyt + Eetyt + P\Eet_xyt => 7o = tf |7i + °2 + P\(ax+px)a2 (2.30)
unit circle. Henee, the {ps} sequence must be convergent. "l^S^^^logram for an
AR(2) process must be such that pQ = 1 and that px be determin^H^^E28). These two Eyty(-\ = axEyt_xyt_x + Eetyt_x + PxEet_xyt_x => -yx = ax-y0 + pxa2 (2.31)
valúes can be viewed as initial valúes for the second-order di^S^^pquation (2.29). E
The fourth panel on the left-hand side of Figure 2.2 ^9^n^ ACF for the yty{-2 ==a\Eyt-\yt-2 +Estyt-i +P\Eet-\yt-2 =>i2 = a
a\ (2-32)
process yt — 0.1yt_\ — 0A9yt_2 + et- ^e properties of the V£úÍBflfE^o^ow directly
from the homogeneous equation yf — 0.7y,_j + 0A9yt_2 — O^^BB^ps are obtained
from the solution to «H|; fy/yt-s= a\Eyt-\yts + Eetyts + P\E^t~\yts => % = aas-\ (2-33)
a = {0.7 ± [(-0.7)2 - 4(0.49)] 1/2}/2fBBft Solving (2.30) and (2.31) simultaneously for 7 0 and jx yields
Since the discriminant d = (-0.7)2 - 4(0.49) is negative^HHRacterístic roots
are imaginary so that the solution oscillates. However, since"^M^|B^9. the solution
1 + ^ + 2^/3, 2
70=
is convergent and the {yt} sequence is stationary. !^HÍMB'' (l-fl?) °
Finally, we may wish to fínd the autocovariances rather t^HBRitocorrelations.
Since we know all of the autocorrelations, if we can fínd thé^MIP ofyt (i.e., 70), (1 + a x p x ) ( a x + P x ) n
we can fínd all of the other js. To fínd 70, use (2.25) and notM|mft= T//To SO tnat 1
^ ------- Tr~~i\ ------a~
To(l ~a\P\ ~^Jh)= o2 JuBi
Substitution for p} and ¿^ yields ^9 H B Henee,
P l = < ' + «. / y <«■+/?■> (2 . 34)
7o=var(,r)=[(,-.2)/(1+a2)] (g|+fl2_;(» (1+^ + 20,/?,)
and ps = ¿í1p5_1 for all s > 2.
Thus, the ACF for an ARMA(1, 1) process is such that the magnitude of px depends
on both ax and /i,. Beginning with this valué of /;,, the ACF of an ARMA(1, 1) process
 THE PARTIAL AUTOCORRELAT1ON FUNCTION 65
64 CHAPTER 2 STATIONARY TIME-SERIES MODELS

The previous p valúes can be treated as initial conditions that satisfy the
loo'ks like that of the AR(1) process. If 0 < ¿7j < 1, convergence will be direct and if-1 <
Yule—Walker equations. For these lags, the shape of the ACF is determined by the
ax < 0, thc autocorrelations will oscillate. The ACF for the function yt = -0.7y,_i + et -
characteristic equation.
0.7£f_] is shown as the last graph on the left-hand side of Figure 2.2. The top portion of
Worksheet 2.2 derives these autocorrelations.
We leave you with the exercise of deriving the correlogram of the ARMA(2, 1) 8. THE PARTIAL AUTOCORRELATION FUNCTIQN
process used in Worksheet 2.1. You should be able to recognize the point that the cor-
In an AR( 1) process,^ andj^_ 9 are cotrelated even thoughyt_2 does not directly appear
relogram can reveal the pattern of the autoregressive coefficients. For an ARMA (p,
in the model. The correlation between yt and yt_i (i.e., p^) is equal to the correlation
g) model beginning after lag g, the valúes of the p¡ will satisfy
between yt andy^j (i.e., p{) multiplied by the correlation betweenyt_x andy/_2 (i.e., px again)
P¡ = a\Pi-\ + alPi-2 + • • • + apPi-p so that />> = (p\)2- lt is important to note that all such indirect correlations are present in
the ACF of any autoregressive process. In contrast, the partial autocorrela-tion
between y¡ and y¡_s eliminates the effeets of the intervening valúes yt_¡ through
V,_v+l- As such, in an AR(1) process the partial autocorrelation between yt andy/-2 ls
equal to zero. The most direct way to find the partial autocorrelation function is to fírst
form the series {y*} by subtracting the mean of the series (i.e., //) from each observa-
CALCULATION OF THE PARTIAL A UTOCORRELATIONS OF tion to obtainy* = yt - /¿. Next, fonn the first-order autoregression
y\ = h i - y /-i + e t
where et is an error temí.
Step 1: Calcúlate the autocorrelations. Use (2.34) to calcúlate p\ as
1
Here the symbol {et} is used since this error process may not be white noise.
(l + 0.49)(-0.7-0.7) = _ Q8445 Since there are no intervening valúes, 4>x { is both the autocorrelation and the partial
1 + 0.49 + 2(0.49) autocorrelation between y, andy^j. Now form the second-order autoregression equation
The remaining autocorrelations decay at the rate p¡ ~ —0.7/? so that P2 = 0.591, P2 = - y* = ^2i>V-l + ^22-yr-2 + et
0.414, p4 = 0.290, p5 = -0.203, p6 = 0.142, p1 = -0.099, /% = 0.07
Here <^S9 is the partial autocorrelation coefficient between yt and yt_2. In other
Step 2: Calcúlate the fírst two partial autocorrelations using (2.35) and (2.36). Henee 4>\\ = words, cf>22 ÍS tne correlation between yt and yt_2 controlling for (i.e., "netting out") the
P\ = -0-8445 4>22 = [0.591 - (-0.8445)2]/[l - (-0.8445)2] = -0.426 effect of yt_\. Repeating this process for all additional lags s yields the partial
Step 3: Construct all remaining <&ss iteratively using (2.37). To find (fe3, note that (fy>\ = &U autocorrelation function (PACF). in practice, with sample size T, ocly TÍA lags are
~ $22^11 =—1-204 and form used in obtaining the sample PACF.
2 2 ~~l
Since most statistical computer packages perform these transformations, there
is little need to elabórate on the computational procedure. However, it should be
pointed out that a simple computational method relying on the so -called Yule-
Walker equations is available. One can form the partial autocorrelations from the
- [-0.414 - (-1.204X0.591) - (-0.426)(-0.8445)]/[l - (-1.204)(-0.8445)
~ (-0.426X0.591)] = autocorrelations as
-0.262 4\\=P\ (2-35)
Similar]y, to find ^44 use ^22 =(/>2-/ 7 f) / ( l -/^f) (2-36)
3 3 - 1
and for additional lags,
<7>44 = PA - Yl'hjPt-j 1 - Yl^jPj s-\ Ps-YlVs-
X.jPs-j
Since a^j = <p2j - </>22<p2¿-j> Jt
f°l l o ws tnal
03i == —1 . 3 1 5, and ^ =
-0-74. Henee
0w = ------- 1± --------------- , 5 = 3,4,5,... (2.37)
^44 =-0.173
If we continué in this fashion. it is possible to demónstrate that <z> 55 = -0.117. 0 66 = -
0.081, ¿77 = -0.056 and <p m = -0.039
where <%=<&_!,,-- <t> S s^s-\^J = U 2, 3, ..., .v-1.
66 CHAPTER 2 STATIONARYTIME-SERIES MODELS SAMPLE AUTOCORRELATIONS OF STATIONARY SERIES 67

For an AR(p) process, there is no direct correlation between yf and y,_s for s > p. equation (p¡ = axPi_x + ai/)^2 + ... + apPi_p). Since the characteristic roots are
Henee, for s > p, all valúes of </>55 will be zero and the PACF for a puré AR(p) process inside the unit circle, the autocorrelations will decay after lag q. Moreover,
should cut to zero for all lags greater than p. This is a useful feature of the PACF that the pattern of the autocorrelation coefficients will mimic that suggested by
can aid in the identiilcation of an AR(p) model. In contrast, consider the PACF for the the characteristic roots.
MA(1) process: y, = £, + 0e,-\- As long as 0 ^ -1, we can writev, / (1 + (3L) = e,, 2. The PACF of an ARMA(/?, q) process will begin to decay after lagp. After
which we know has the inñnite-order autoregressive representation lag/?, the coefTicients of the PACF (i.e., the </>ss) will mimic the ACF coef-
y, - &y,-\ + i32y>i-2 - P3y*-3 + • • • = e¡ ficients from the model yt/(I +/?,¿ + fol2 + ... + @ ¿<i).
As such, the PACF will not jump to zero sincej^ will be correlated with all of its We can i I lústrate the usefulness of the ACF and PACF functions using the model
own lags. Instead, the PACF coefficients exhibit a geometrically decaying pattem. If y, = cio + 0.1y(_i + £,. If we compare the top two graphs in Figure 2.2, the ACF shows
0 < 0, decay is direct and if f3 > 0, the PACF coefficients oscillate. the monotonic decay of the autocorrelations while the PACF exhibits the single spike
Worksheet 2.2 illustrates the procedure used in constructing the PACF for the at lag 1. Suppose that a researcher collected sample data and plotted the ACF and
ARMA(1, 1) model shown in the fífth panel on the right-hand side of Figure 2.2: PACF functions. If the actual patterns compared favorably to the theoretical patterns,
y t = -0.7^_! + €, - 0.7e M the researcher might try to estímate data using an AR(1) model. Correspondingly, if
First, calcúlate the autocorrelations. Clearly, p0 = 1; use equation (2.34) to calcú- the ACF exhibited a single spike and the PACF exhibited monotonic decay (see the
late as p] = -0.8445. Thereafter, the ACF coefficients decay at the rate p¡ = (-0.1)p¡_i third graph for the model yt = et - 0.7£>_,) the researcher might try an MA(1) model.
for / > 2. Using (2.35) and (2.36), we obtain cpu = -0.8445 and (p22 =-0.4250. All 7. SAMPLE AUTOCORRELATIONS OF
subsequent </>ss and (f>sj can be calculated from (2.37) as in Worksheet 2.2.
More generally, the PACF of a stationary ARMA(p, q) process must ultimately STATIONARY SERIES __________________________
decay toward zero beginning at lagp. The decay pattern depends on the coefficients of In practice, the theoretical mean, variance, and autocorrelations of a series are
the polynomial (1 + f3xL + 02Ll + • • • + PqLq)- Table 2.1 summarizes some of the proper-ties unknown to the researcher. Given that a series is stationary, we can use the sample
of the ACF and PACF for various ARMA processes. Also, the right-hand side graphs of mean, variance and autocorrelations to estímate the parameters of the actual data
Figure 2.2 show the partial autocorrelation ñinctions of the fíve indicated processes. generating process. Let there be T observations labeled^ through yT. We can iet y,
For stationary processes, the key points to note are the following: <r2, and rs be estimates of ¿¿, o2, and ps respectively where1
1. The ACF of an ARMA(p, q) process will begin to decay after lag q. After T
lag q, the coefficients of the ACF (i.e., the p¡) will satisfy the difference y=(l/T)J2yí (2.38)
T

Table 2.1 Properties of the ACF and PACF a2 = (i/T)J2(yt-y)2 (2.39)

Process ACF PACF and for each valué of .y = 1, 2, ....
T
White noise All ps = 0 (s =* 0) All <pss = 0
AR(1): a-, > 0 Direct exponential decay: p s - af cp^ = /?-,; <^ss = 0 for s > 2 E (yt-yXyt-s-y)
AR(1): a-, < 0 Oscillating decay: ps = af <f>^ = p,; éss = 0 for s > 2 rs ------------------f ------------------ (2.40)
AR(p) Decays toward zero. Coefficients Spikes through lag p. All éss = 0 2
may oscillate. for s > p.
Ecv(->)
MA(1): ¡3-> 0 Positive spike at lag 1. ps - 0 for s > 2 Oscillating decay: <?-,, > 0.
The sample autocorreiation function [i.e., the ACF derived from (2.40)] and the
MA(1): 0 < 0 Negative spike at lag 1. ps = 0 for s > 2 Geometric decay: <>-,-, < 0.
sample PACF can be compared to various theoretical functions to help identify the
ARMA(1, 1): Exponential decay beginning at lag 1. Oscillating decay beginning at
actual nature of the data generating process. Box and Jenkins (1976) discuss the dis-
ay > 0 Sign p-, = sign(a-, + 3) lag 1. ci11 = py
ARMA(1, 1): Oscillating decay beginning at lag 1. Exponential decay beginning at
tribution of thesample valúes of>5 under the nuil that v, is stationary with normally
a-i < 0 Sign p-, = sign(a-, + LÍ) lag 1. o^ - py and sign{<pss) distributed errors. Allowing var(rvj to denote the sampling variance of>5, they obtain
= signan). Var(ri.)=7^1 for s - 1
ARMA (p, q) Decay (either direct or oscillatory) Decay (eíther direct or oscil- s-\
beginning at lag q. latory) after lag p. -r~ l (n-2^,.2) fo rv > , (241)
7=1
68 CHAPTER 2 STATI0NANYTIME-SER1ES MODELS SAMPLE AUTOCORRELATIONS OF STATIONARY SERIES 69

if the true valué oí rv = 0 [i.e., if the true data-generating process is an MA(s-l) Henee, using the residuals of an ARMA(y;, q) model, Q has a \ 2 with A- - p — q deerees
process]. Moreover, in large samples (i.e., for large valúes of T), rs will be normally of freedom ( i f a constant is included, the degrees of freedom are .y ~p — q — \\
distributed with a mean equal to zero. For the PACF coeffícients, under the nuil
hypothesis of an AR(p) model (i.e., under the nuil that all <f>p+itP-n are zero), the vari- IVlodei Selection Gritería
anee of the </>p+i,p+i is approximately 1/7".
In practice, we can use these sample valúes to form the sample autocorrelation One natural question to ask of any estimated model is: How well does it fit the data?
and partial autocorrelation functions and test for significanee using (2.41). For exam- Adding additional lags for/? and/or q will necessarily reduce the sum of squares of
ple, if we use a 95 percent confidence interval (i.e., two standard deviations), and the the estimated residuals. However, adding such lags entails the estimation of addi-
calculated valué of rx exceeds 2r~l/2, it is possible to reject the nuil hypothesis that the tional coeffícients and an associated loss of degrees of freedom. Moreover, the inclu-
fírst-order autocorrelation is not statistically different from zero. Rejecting this sión of extraneous coeffícients will reduce the Ibrecasting performance of the fitted
hypothesis means rejecting an MA(s - 1) = MA(O) process and accepting the alterna- model. As discussed in some detail in Appendix 2.2 of this chapter, there exist vari-
tive q > 0. Next, try 5 = 2; var(r2) is: (l+2rj2)/r. If rx is 0.5 and T is 100, the variance of ous model selection cnteria that trade ofY a reduction in the sum of squares of the
r2 is 0.015 and the standard deviation is about 0.123. Thus, if the calculated valué of r2 residuals for a more parsimonious model. The two most commonly used model
exceeds 2(0.123), it is possible to reject the hypothesis r2 = 0. Here, rejecting the nuil selection cnteria are the Akaike Information Criterion (A1C) and the Schwartz
means accepting the alternative that q > 1. Repeating for the various valúes of s is Bayesian Criterion (SBC). Although there are several different ways to report the cri-
helpful in identifying the order to the process. The máximum number of sample teria (as illustrated by question 10 at the end of this chapter), all will select the same
autocorrelations and partial autocorrelations to use is typically set equal to TÍA. model. In the text, we will use the following formulas
Within any large group of autocorrelations, some will exceed two standard devi- AIC = T In (sum of squared residuals) + 2n SBC =
ations as a result of puré chance even though the true valúes in the data-generating T In (sum of squared residuals) + /? ln(7)
process are zero. The g-statistic can be used to test whether a group of autocorrela- w/wre: n — number of parameters estimated (p + q + possible constant term) T—
tions is signifícantly different from zero. Box and Pierce (1970) used the sample auto- number of usable observations
correlations to form the statistic When you estímate a model using lagged variables, some observations are lost.
Q = T±rl To adequately compare the alternative models, Tshould be kept fixed. Otherwise, you
will be comparing the performance of the models over different sample periods.
Moreover, decreasing T has direct effect of reducing the AIC and the SBC; the goal
Under the nuil hypothesis that all valúes ofrk = 0, Q is asymptotically x2 distrib- is not to select a model because it has the smallest number of usable observations. For
uted with s degrees of freedom. The intuition behind the use of the statistic is that example, with 100 data points, estimate an AR(1) and an AR(2) using only the last 98
high sample autocorrelations lead to large valúes of Q. Certainly, a white-noise observations in each estimation. Compare the two models using T— 98.
process (in which all autocorrelations should be zero) would have a Q valué of zero. Ideally, the AIC and SBC will be as small as possible (note that both can be neg-
If the calculated valué of Q exceeds the appropriate valué in a x2 table, we can reject ative). As the fit of the model improves, the AIC and SBC will approach —oo. We can
the nuil of no signifícant autocorrelations. Note that rejecting the nuil means accept- use these cnteria to aid in selecting the most appropriate model; model A is said to fit
ing an alternative that at least one autocorrelation is not zero. better than model B if the AIC (or SBC) for A is smaller than for model B. In using
A problem with the Box-Pierce Q-statistic is that it works poorly even in mod- the criteria to compare alternative models, we must estimate them over the same sam-
erately large samples. Ljung and Box (1978) report superior small sample perform- ple period so that they will be comparable. For each, increasing the number of regres-
ance for the modified Q-statistic calculated as sors increases n but should have the effect of reducing the sum of squared residuals.
Thus, if a regressor has no explanatory power, adding it to the model will cause both
2^r(7 + 2)¿ r 2/(T -/c) (2.42) the AIC and SBC to increase. Since In(7) will be greater than 2, the SBC will always
select a more parsimonious model than will the AÍC; the marginal cost of adding
If the sample valué of Q calculated from (2.42) exceeds the critical valué of x2 witn s regressors is greater with the SBC than with the AIC.
degrees of freedom, then ai least one valué of rk is statistically different from zero at the Of the two criteria, the SBC has superior large sample properties. Let the true order
specifíed significance level. The Box-Pierce and Ljung-Box g-statistics also serve as a of the data generaling process be (//, q*) and suppose that we use the AIC and SBC to
check to see if the residuals from an estimated ARMA(p, q) model behave as a white- estimate all ARMA models of order {p, q) where/p > p* and q > q*. Both the AIC and
noise process. However, when the s correlations from an estimated ARMA(/?5 q) model the SBC will select models of orders greater than or equal to (p*, q*) as the sample size
are formed, the degrees of freedom are reduced by the number of estimated coeiíícients. approaches infínity. However, the SBC is asymptotically consistent while the AIC is
70 CHAPTER2 STATIONARYTiME-SERIES MODELS

biased toward selecling an overparameterized model. However, in small samplcs, the

AIC can work better than the SBC. You can be quite confidenl in your results if both
the AIC and the SBC select the same model. If they select different modeis, you need
to proceed cautiously. Since SBC selects the more parsimonious model, you should
check to determine if the residuals appear to be white noise. Since the AIC can select
an overparameterized model, the /-statistics of all coefTicients should be significant at
conventional levéis. A number of other diagnostic checks that can be used to compare
alternative modeis are presented in Sections 8 and 9. "Nevertheless, it is wise to retain a
healthy skepticism of your estimated modeis. With many data sets, it is just not possi-
ble to fmd the one model that clearly dominates all others. There is nothing wrong with
reporting the results and the forecasts using alternative estimations.

Estimation of an AR{1) Model

Let us use a specific example to see how the sample autocorrelation function and partial
autocorrelation function can be used as an aid in identifying an ARMA model. A com-
puter program was used to draw 100 normally distributed random numbers with a theo-
retical variance equal to unity. Cali these random variates et, where / runs from I to 100.
Beginning with t - I, valúes oíyt were generated using the formulayt — 0.7>>r_] + et and
the initial condition^o = 0. Note that the problem of nonstationarity is avoided since the
initial condition is consistent with long-run equilibrium. The upper-left-hand graph of
Figure 2.3 shows the sample correlogram and the upper-right-hand graph shows the sam-
ple PACF. You should take a minute to compare the ACF and PACF to those of the the-
oretical processes shown in Figure 2.2.
In practice, we never know the true data-generating process.. However, suppose
we were presented with these 100 sample valúes and were asked to uncover the true autoregressive model. Nevertheless, if we did not know the true underlying
process. The first step might be to compare the sample ACF and PACF to those of the process, and happened to be using monthly data, we might be concerned with the
various theoretical modeis. The decaying pattern of the ACF and the single large spike significant partial autocorrelation at lag 12. After all, with monthly data we might
at lag I in the sample PACF suggests an AR(l) model. The first three autocorrelations expect some direct relationship between y, and V/_p.
are rx = 0.74, r2 = 0.58, and r3 = 0.47 (which are somewhat greater than the theoretical Although we know that the data was actually generated from an AR(1) process,
valúes of 0.7, 0.49 (0.72 = 0.49), and 0.343. In the PACF, there is a sizable spike of it is illuminating to compare the estímales of two different modeis. Suppose we estí-
0.74 at lag I, and all other partial autocorrelations (except for lag 12) are very small. mate an AR( 1) model and also try to capture the spike at lag 12 with an MA coeffi-
Under the nuil hypothesis of an MA(0) process, the standard deviation of rx is cient. Thus, we can consider the two tentative modeis
T~^12 = o.l. Since the sample valué of/-] = 0.74 is more than seven standard devia- Model \ : y t = auvt_{ + et
tions from zero, we can reject the nuil hypothesis that r] equals 0. The standard devi- Model 2: v, = a x y t _ x + s t + V^t-n*
ation of r2 is obtained by applying (2.41) to the sampling data, where s = 2:
Table 2.2 reports the results of the two estimations. The coeffícient of model 1
Var(r2) = (1 + 2(0.74)2)/100 = 0.021 satisiies the stability condition | ¿7, | < 1 and has a lovv standard error (the associated f-
1/2
Since (0.021) = 0.1449, the sample valué of r2 is more than three standard statistic for a nuil of zero is more than 12). As a use ful diagnostic check, we plot the
deviations from zero; at conventional signifícance levéis, we can reject the nuil correlogram of the residuals of the Htted model in Figure 2.4. The g-statistics for
hypothesis that r2 equals zero. We can similarly test the sfgnificance of the other val- these residuals indícate that each one of the autocorrelations is less than 2 standard
úes of the autocorrelations. deviations from zero. The Ljung-Box O-statistics of these residuals indícate that as
As you can see in the second panel of the figure, other than o6n, all partial "groitp. lags 1 through 8, 1 through 16, and I through 24 are not signifícantly differ-
autocorrelations (except for lag 12) are less than^r" 172 = 0.2. The decay of the ent from zero. This is strong evidence that the AR(1) model ktfits" the data well. After
ACF and the single spike of the PACF give the strong impression of a first -order a
'l, ií residual autocorrelations were significant, the AR(1) model would not utilize
72 CHAPTER 2 STATIONARYTIME-SERIES MODELS SAMPLE AUTOCORRELATIONS OF STATIONARY SERÍES 73

Table 2.2 Estimates of an AR(1) Modei Estimatiomt cf an ARMA(1, 1) Mode!

Model 1 Model 2 A second {y,} sequence was constructed to ¡Ilústrate the estimation of
Yt = ai Kt-i + ¿t Kt = ai Kt-i + et + /312^f-i2 ARMA(1, 1). Gíven 100 normally distributed valúes of {f,}, 100 valúes of { y \
Degrees of freedom 99 98
were generated using '
Sum of squared residuals 85.21 85.17 y^-0.7^1 +^-0.7^j
Estimated a, 0.7910 0.7953 where >'o and e0 were both set equal to zero.
(standard error) (0.0622) (0.0638)
Both the sample ACF and the PACF from the simulated data (see the second set
Estimated fl -0.033
(standard error) (0.1134) of graphs in Figure 2.3) are roughly equivalent to those of the theoretical model
AIC; SBC AIC = 441.9; SBC = 444.5 AIC = 443.9; SBC = 449.1 shown in Figure 2.2. However, if the true data-generating process were unknown, the
Ljung-Box ü-statistics for Q(8) = 6.43 (0.490) Q(8) = 6.48 (0.485) researcher might be concerned about certain discrepancies. An AR(2) model could
the residuals (significance Q(16) = 15.86 (0.391) Ü(16) = 15.75 (0.400) yield a sample ACF and PACF similar to those in the figure. Table 2.3 reports the
ievel in parentheses) O(24) = 21.74 (0.536) O(24) == 21.56 (0.547) results of estimating the data using the following three models:
Model \ : y , = a x y t _ x + e t
Model 2: yt = axy(_x + et + /?,£-,_,
Model 3: y, = a x y t _ x + a^^ + e (
In examining Table 2.3, notice that ali of the estimated valúes of ax are highly
significant; each of the estimated valúes is at least eight standard deviations from
zero. It is clear that the AR(1) model is inappropriate. The Q-statistics for model 1
indícate that there is significant autocorrelation in the residuals. The estimated
ARMA(1, i) model does not suffer from this problem. Moreover, both the AIC and
the SBC select model 2 over model 1.
The same type of reasoning indi cates that model 2 is preferred to model 3. Note
that for each model, the estimated coefficients are highly significant and the point
estimates imply convergence. Although the Q-statistic at 24 lags indicates that these

Table 2.3 Estimates of an ARMA (1, 1) Model

Estimates1 Q-Statistics2 AIC/SBC3

all available information concerning movements in the {y¡} sequence. For example,
suppose we wanted to forecast >'í+1 conditioned on all available information up to and Mode! 1 a,: -0.835 (.053) ü(8) = 26.19 (.000) AIC = 496.5
including period /. With model 1, the valué of y{+x is yt+x = axyt + et+x. Henee, the Q(24) = 41.10 (.001) SBC = 499.0
forecast from model 1 is axyt. If the residual autocorrelations liad been significant, Model 2 a,:-0.679 (.076) Q(8) = 3.86 (.695) AIC = 471.0
this forecast would not capture all of the available information set. £,: -0.676 (.081) Q(24) = 14.23 (.892) SBC = 476.2
Examining the results for model 2, note that both models yield similar estimates Model 3 a-,:-1.16 (.093) Q(8) = 11.44 (.057) AIC = 482.8
for the first-order autoregressive coeffícient and the associated standard error. a2: -0.378 (.092) Q(24) = 22.59 (.424) SBC = 487.9
However, the estimate for j3Xi IS of poor quality; the insignificant t valué suggests Notes:
that it should be dropped from the model. Moreover, comparing the AIC and the Standard errors ¡n parentheses.
SBC valúes of the two models suggests that any benefits of a reduced sum of Ljung-Box Q-statisties of the residuals from the fitted model. The
squared residuals is overwhelmed by the detrimental effeets of estimating an addi- significance levéis are in parentheses.
tional parameter. All of these indicators point to the choice of model 1. For comparability, the AIC and SBC valúes are reported for estimations
Exercise 7 at the end of this chapter entails various estimations using this data that used only observations 3 through 100. If the AR(1) is estimated using
99 observations, the AIC and SBC are 502.3 and 504.9, respectively. If the
set. In this exercise you are asked to show that the AR(1) model performs better than ARMA(1, 1) is estimated using 99 observations, the AIC and SBC are
some alternative specifications. It is important that you complete this exercise. 476.6 and 481.1, respectively.
 SAMPLE AUTOCORRELATIONS OF STATIONARY SERIES 75
74 CHAPTER 2 STATIONARYTIME-SERIES MODELS

All estimated coefficients are significant and the Ljung-Box g-statistics for the
two models do not suffer from correlated residuals, the g-statistic at 8 lags indícales residuals are all ¡nsigniflcant at conventional levéis. In conjunction with the fact that
serial correlation in the residuals of model 3. Thus, the AR(2) model does not capture the AIC and SBC both select this second model, the researcher unaware of the true
short-term dynamics as well as the ARMA(1, 1) model. Also note that the AI C and process might be tempted to conclude that the data-generating process includes a
SBC both select model 2. moving average term at lag 16.
A useful model check is to split the sample into two parts. If a coefficient is pres-
Estimation of an AR(2) Model
ent in the data-generating process, its infiuence should be seen in both subsamples. If
A third data series was simulated as the simulated series is split into two parts, the ACF and PACF using observations 50
through 100 follow:
y, = OJy^i ~ 0A9y t _ 2 + e t
The estimated ACF and the PACF of the series follow: A C F ACF
Lags
Lags:
1-12: 0.466 -0.161 -0322 -0.108 -0.052 -0.165 - 1-12: 0.460 -0.207 -0.280 0.035 0.095 -0.153 -
0.010 0.128 0.180 0.034 -0.087 -0.113 0.133 0.102 0.181 0.027 -0.009 0.009
13-24: -0.164 -0.058 0.115 0.254 0.046 -0.175 13-24: -0.058 -0.088 0.042 0.205 0.064 -0.162 -
0.150 0.010 0.03.2 -0.089 -0.046 0.052 0.184 -0.050 -0.033 -0.145 -0.107 -0.036
PACF PACF
1-12: 0.466 -0.482 0.023 0.045 -0.253 -0.121 1-12: 0.460 -0.531 0.193 0.063 -0.197 -0.130
0.101 0.037 -0.076 0.023 -0.020 -0.139 0.234 -0.078 0.004 0.065 0.154 -0.257
13_24: -0.167 0.207 0.007 0.085 -0.216 0.013 - 13-24: 0.026 0.146 0.038 0.004 -0.054 -0.009 -
0.022 -0.032 0.015 -0.061 0.038 -0.184 0.137 -0.076 -0.034 -0.083 -0.029 -0.220
Note the large autocorrelation at lag 16 and the large pardal autocorrelations at As you can see, the size of the partial autocorrelations at lags 14 and 17 is
lags 14 and 17. Given the way the process was simulated, the presence of these auto - diminished. Now, estimating a puré AR(2) model over this second part of the sample
correlations is due to nothing more than chance. However, an econometrician yields
unaware of the actual data-generating process might be concerned about these auto -
correlations. The coefficients of the AR(2) model are estimated as Coefficient Estímate Standard Error /-Statistic Significance Level

Coefficient Estímate Standard Error /-Statistlc Signifícance Level ax 0.713855785 0.120541523 5.92207 0.00000031
a2 -0.537843744 0.120420318 -4.46639 0.00004687
ax 0.692389807 0.089515769 7.73484 0.00000000
a2 -0.480874620 0.089576524 -5.36831 0.00000055 0(8) = 7.83, significance level 0.251
____________ AIC = 219.87333 _________ SBC = 225.04327 ___________________ g(16) = 15.93, significance level 0.317
________ 0(24) = 26.06, significance level 0.249 _____________________________
Overall, the model appears to be adequate. However, the two AR(2) coefficients
are unable to capture the correlations at very long lags. For example, the partial auto- All estimated coefficients are significant, and the Ljung-Box g-statistics do not
correlations of the residuals for lags 14 and 17 are both greater than 0.2 in absolute indícale any significant autocorrelations in the residuals. In fact, this model does cap -
valué. The ealculated Ljung-Box statistic for 16 lags is 24.6248 (which is significant ture the actual data-generating process quite well. In this example, the large spurious
at the 0.038 level). At this point, it might be tempting to try to model the correlation at autocorrelations of the long lags can be eliminated by changing the sample period.
lag 16 by including the moving average term 0\^s J _ x ^. Such an estimation results in 2 Thus, it is ha.rd to maintain that the correlation at lag 16 is meaningful. Most sophis -
ticated practitioners warn against trying to fit any model to the very long lags. As you
Coefficient Estímate Standard Error r-Statistic Significance Level can infer from (2.41), the variance of rs can be sizable when s is large. Moreover, in
ax 0.716681247 0.091069451 7.86961 0.00000000 small samples, a few "unusual" observations can créate the appearance of significant
a2 -0.464999924 0.090958095 -5.11224 0.00000165 autocorrelations at long lags. Since econometric estimation involves unknown'data-
ft6 0.305813568 0.109936945 2.78172 0.00652182 generating processes, the more general point is that we al way s need to be wary of our
___________ AIC = 213.40055 _________ SBC = 221.15545 ___________________ estimated model. Fortunately, Box and Jenkins (1976) established a set of procedures
that can be used to check a model's adequacy.
 BOX-JENKINS MODEL SELECTION 77
76 CHAPTER 2 STATIONARYTIME-SER1ES MODELS

8. BOX-JENKINS MODEL SELECTION If you passcd the last quiz, yon know that (1 - 0.25L2)yf = (1 + 0.5L)et is equivalent
The estimates of the AR(1), ARMA(1, 1) and AR(2) models in the previous section to (1 + 0.5¿)( 1 - 0.5L)yt = (1 + 0.5¿)¿-, so that y, = 0.5 v,_, + st. In practice, the
illustrate the Box-Jenkins (1976) strategy for appropriate model selection. Box and polynomials wi l l not factor exactly. However, if the factors are similar, you should try
Jenkins popularized a three-stage method aimed at selecting an appropriate model for a more parsimonious form.
the purpose of estimating and forecasting a univariate time series. In the identifica -lion ln order to ensure that the model is parsimonious, the various a¡ and jtf¡ should all
stage, the researcher visually examines the time plot of the series, the autocor-relation have /-statistics of 2.0 or greater (so that each coefficient is signiflcantly different
function, and the partial correlation function. Plotting the time path o f the {yi) from zero at the 5% level). Moreover, the coefficients should not be strongly corre -
sequence provides useful information concerning outliers, missing valúes, and lated with each other. Highly collinear coefficients are unstable; usually one or more
structural breaks in the data. TMonstationary variables may ha ve a pronounced trend or can be climinated from the model without reducing forecast performance.
appear to meander without a constant long-run mean or variance. Missing valúes and
outliers can be corrected at this point. At one time, the standard practice was to first- Stationarity and Invertibility
difference any series deemed to be nonstationary. Currently, a large literature is
The distribution theory underlying the use of the sample ACF and PACF as approxi-
evolving that develops formal procedures to check for nonstationarity. We defer this
mations to those of the true data-generating process assumes that the {y(} sequence is
discussion until Chapter 4 and assume that we are working with stationary data. A
stationary. Moreover, /-statistics and <2~statistics also presume that the data are sta-
comparison of the sample ACF and PACF to those of various theoretical ARMA
tionary. The estimated autoregressive coefficients should be consistent with this
processes may suggest several plausible models. In the estimation stage, each of the
underlying assumption. Henee, we should be suspicious of an AR(1) model if the esti-
tentative models is fit and the various a¡ and /3¡ coefficients are examined. In this sec-
mated valué of ¿71 is cióse to unity. For an ARMA(2, q) model, the characteristic roots
ond stage, the estimated models are compared using the following criteria.
of the estimated polynomial (1 - a¡L — a2L2) should lie outside of the unit circle.
Parsimony As discussed in greater detail in Appendix 2.1, the Box—Jenkins approach also
A fundamental idea in the Box-Jenkins approach is the principie of parsimony. necessitates that the model be invertible. Formally, {yf} is invertible if it can be rep-
Parsimony (meaning sparseness or stinginess) should come as second nature to econ- resented by a finite-order or convergent autoregressive process. Invertibility is impor-
omists. Incorporating additional coefficients will necessarily increase fit (e.g., the tant because the use of the ACF and PACF implicitly assume that the {y{} sequence
valué of R2 will increase) at a cost of reducing degrees of freedom. Box and Jenkins can be represented by an autoregressive model. As a demonstration, consider the sim-
argüe that parsimonious models produce better forecasts tharí overparameterized ple MA(1) model:
models. A parsimonious model fits the data well without incorporating any needless v, = £-, -/?,£■;_, (2.45)
coefficients.,.The aimis to ápproximate the true data-generating process but not to pin so that if | /?, | < 1,
down the exact process. The goal of parsimony suggested eliminating the MA(12)
coefficient in the simulated AR(1) model shown earlier. y,/(\ -/6 ]L) = e í
In selecting an appropriate model, the econometrician needs to be aware that sev- or
era! different models may have similar properties. As an extreme example, note that y, +/?,>>,_, +0ht-2 +P\yt-3 + • • • = et (2.46)
the AR(1) model yt = 0.5yt_] + s¡ has the equivalent infinite-order moving-average If | /Vj I < 1, (2.46) can be estimated using the Box—Jenkins method. However,
representation ofyt = et + 0.5^_! + 0.25^_2 + 0.125^_3 + 0.0625ef_4 + .... In most 'H P\ I > U the \yt\ sequence cannot be represented by a finite-order AR process;
samples, approximating this MA(co) process with an MA(2) or MA(3) model will as such, it is not invertible. More generally, for an ARMA model to have a conver-
give a very good fit. However, the AR(1) model is the more parsimonious model and gent AR representation, the roots of the polynomial (1 + (3XL + f^L2 + ... +'/3gL?)
is preferred. As a test, you should show that this AR(1) model has the equivalent rep- niust lie outside of the unit circle. Note that there is nbthing improper about a non-
resentation ofy, = 0.25vr_2 + 0.5eM + ev invertible model. The {yf} sequence implied by yt = et - et_\ is stationary in that it
Also, be aware of the common factor problem. Suppose we wanted to fit the has a constant time-invariant mean [Eyt = Eyt_s — 0], a constant time-invariant variance
ARMA(2, 3) model [var(y¡) — var(v7_5) = ÍT2( I + 6 \ ) + 2a2], and the autocovariances 7j = -/^ o2- and all
(1 - axL - a2L2)yt = (1 + (3XL + fcl2 + &l?)et (2.43) other 7y = 0. The problem is that the technique does not allow for the estimation of
Suppose that (1 - axL - o2L2) and (1 + f3xL + j^L2 + /%L3) can each be factored as such models. If/?, = 1, (2.46) becomes
(1+ cL){\ + aL) and (1 + cL)(l + bxL + b2L2\ respectively. Since (1 + cL) is a common
factor to each, (2.43) has the equivalent, but more parsimonious, form 3 Clearly, the autocorrelations and partial autocorrelations between yt and^^- will
(1 + aL)yt = (!+/?!! + b2L2)et (2.44) "ever decay.
78 CHAPTER2 STATIONARYTIME-SERIES MODELS
PROPERT1ES OF FORECASTS 79

Goodness of Fit Let the sum of the squared residuals from each model be SSR l and SSR
A good model will fu the data well. Obviously, R2 and the average of the residual sum respectively. To test the restriction that afl coeffícients are equal [i.e., a Q ( 1 ) = a (2)
of squares are common goodness-of-fií measures in ordinary least squares. The prob- and < 7 , ( 1 ) = ¿;,(2) and ... a p{\) = a p {2) and /*,( 1) = /?,(2) and ... 0 q (\) = ^(2)],°use an
lem with these measures is that the fit necessarily improves as more parameters are F-test and fo-rm5
included in the model. Parsimony suggests using Ihe A1C and/or SBC as more appro- F _(SSR-SSRl -SSR2 )/n
priate measures of the overall fit of the model. Aiso be cautious of estimates that fail {SSR\+SSR2)'(T-2n)
to converge rapidly. JVlost software packages estímale the parameters of an ARMA
where n - number of parameters estimated (n = p + q + 1 if an intercept is included
model using a nonlinear search procedure. If the search fails to converge rapidly, it is
and p + q otherwise) and the number of degrees of freedom are (n, T- 2n).
possible that the estimated parameters are unstable. In such circumstances, adding an Intuitively, if the restriction is not binding (i.e., if the coeffícients are equal), the
additional observation or two can greatly alter the estimates. sum SSR, + SSR2 should equal the sum of the squared residuals from the entire sam-
The third stage of the Box-Jenkins methodology involves diagnostic checking. ple estimation. Henee, F should equal zcro. The larger the calculated valué of F, the
The standard practice is to plot.the residuals to look for outliers and for evidence of more restrictive is the assumption that the coeffícients are equal.
periods in which the model does not fit the data well. If alí plausible ARMA models
Similarly, a model can be estimated over only a portion of the data set. The esti-
show evidence of a poor fit during a reasonably long portion of the sample, it is wise
mated model can then be used to forecast the known valúes of the series. The sum of
to consider using intervention analysis, transfer function analysis, or any other of the the squared forecast errors is a useful way to compare the adequacy of alternative
multivariate estimation methods discussed in later chapters. If the variance of the models. Those models with poor out-of-sample forecasts should be eliminated. Some
residuals is increasing, a logarithmic transformaron may be appropriate. of the details in constructing out-of-sample forecasts are discussed in the next section.
Alternad vely, you may wish to actually model any tendency of the variance to change
using the ARCH techniques discussed in Chapter 3.«
It is particularly important that the residuals from an estimated model be seri-
9, PROPERT1ES OF FORECASTS ____________
ally uncorrelated. Any evidence of serial correlation implies a systematic move- Perhaps the most important use of an ARMA model is to forecast future valúes of
tne
ment in the {yt} sequence that is not accounted for by the ARMA coeffícients ÍV/} sequence. To simplify the following discussion, it is assumed that the actual
included in the model. Henee, any of the tentative models yielding nonrandom data-generating process and the current and past realizations of the {st} and {yt}
residuals should be eliminated from consideraron. To check for correlation in the sequences are known to the researcher. First consider the forecasts from the AR(1)
residuals, construct the ACF and the PACF of the residuals of the estimated model. model yt = aG + a\)'t-\ + £}• Updating one period, we obtain
You can then use (2.41) and (2.42) to determine whether any or all of the residual y,.H =a() + axyt+ eí+[
autocorrelations or partí al autocorrelations are statistically significant. 4 Although
If you know the coeffícients ÜQ and a,, you can forecast y /+ , conditioned on the
there is no signiflcance level that is deemed "most appropriate," be wary of any
information available at period / as
model yielding (1) several residual correlations that are marginally significant and
(2) a g-statistic that is barely significant at the 10 percent level. In such circum-
stances, it is usually possible to formúlate a better performing model. If there are where Etyí+ j is a short-hand way to write the conditional expectation ofyt+J- given the
sufficient observations, fitting the same ARMA model to each of two subsamples information available at /.
can provide useful information concerning the assumption that the data-generating Formally, Ety[+J= E(yt+J \ yn yt_u y{_2, ..., e>, £>_,, ...).
process is unchanging. In the AR(2) model that was estimated in the last section, In the same way, sincey t+2 = a 0 + a,v /+, + s (+2, the forecast ofy í+2 conditioned on
the sample was split in half. In general, suppose you estimated an ARMA(p, q) the information available at period / is
model using a sample size of T observations. Denote the sum of the squared resid-
uals as SSR. Divide the Tobservations into two subsamples with tm observations in £/J7t-2 = ao + a\Etyt+\ and
the first and tn = T- tm observations in the second. Use each subsample to estímate using (2.48)
the two models E
i. vt+2 = a
o + a
\( a o + a
0't)
v, = tf()(i) + tfi(i)v,-i • • •
+
p( iyf-P t, rt0)z,-) + • • • 4/0)^-<?
+a ] + + + Thus, the forecast of v /+1 can be used to forecast y(+2. The point is that forecasts c^n
using/,, ..../ w be constructed using forward iteration; the forecast ofyt+j can be used to forecast -
*'/</+1- Since.v/+/-+i = Ü() + £7|\'/+y + £>+/+!, it immediately follows that
v, = ao{2) + ^1(2)v/_i + ... + ap(2)yi_p + st + ^1(2)£/_1 + ... + 0q(2)er_^ E
t.vt+j+\ =
"o + a E
\ í}'írj (2.49)
using/w+l, ..., tT
80 CHAPTER 2 STATiONARYTIME-SERiES MODELS

From (2.48) and (2.49) it should be clear that ii is possiblc lo obtain the enlire and the forecast error is o2. As sueh, the 95 percent confidence interval for the one-step-
sequcnce of /-step-ahead forecasts by forward ilcration. Consider E¡yl+j =oQ(\ -±ax 4-ax ahead forecast can be constructed as
4-... + £//~ ) + aJ]yI
This equation, called the forecast function, expresses all of they-step-ahead fore- We can construct a confidence interval for the two-step-ahead forecast error in
casts as a function of the information set in period /. Unfortunately, the quality of the the same way. From (2.49), the two-step-ahead forecast is ao(\ + a x ) +a] y¡ and (2.51)
forecasts declines as we forecast further out into the future. Think of (2.49) as a fírst- indicates that var[<?/(2)] is ^(1 + a}). Thus, the 95 percent confidence interval for the
order difference equation in the {Etyí+j} sequence. Since | ax \ < 1, the diñerence equa- two-step-ahead forecast is
tion is stable, and it is straightforward to Fmd the particular solution to the diiference « o 0 + a \ ) + a h t ± l - 9 6 c r 0 + aj ) l / 2
equation. If we take the limit of Efyí+j asj —► oo, we fmd that Ety1+j —* ao¡(\ ~.ax). This
result is really quite general: For any siationaiy ARMA model, the con ditionaI forecast
ofyf+J- converges to the unconditional mean as j -—» oo. HIgher-Order Podéis
Because the forecasts from an ARMA model will not be perfectly accurate, it is To generalize the discussion, it is possible to use the iterative technique to derive the
important to consider the properties of the forecast errors. Forecasting from time forecasts for any ARMA(p, q) model. To keep the algebra simple, consider the
period ?, we can define the /-step-ahead forecast error—ef(J)—as the difference ARMA(2, 1) model
between the realized valué of yf+j and the forecasted valué
yt = a0 + axyt_x + «2>;/-2 + Et + P\€t-\ (2.52)
e,(J)=yt+j-Etyi+j
Henee, the one-step-ahead forecast error is e¡(\) = yí+\ - Etyt+X = eí+x (i.e., the Updating one period yields
"unforecastable" portion ofyí+] given the information available in t). y í + [ =a o + a x y t + a^y^ + s (+] + (3 X e t

To find the two-step-ahead forecast error, we need to form et(2) = yí+2 — Etyi+2- If we continué to assume that (1) all coefficients are known; (2) all variables sub-
Sinceyí+2 = £?o + aU;r+l ~*~ €t+2 an<^ J^/3;/+2 = a0 + a\Et)rt+\^ ^ follows that scripted /, /-I, t-2, ... are known at period /; and (3) E(£(+J- = 0 for7 > 0, the condi-
tional expectation of>*/+j is
et(2) = ax(yt+l - E^f+i) + ¿r/+2 = et+2 + a\et+\ E
tyt+\ = ao + a\yt+ aiyt-\ + 0\et (2-53)
You should take a few moments to demónstrate that for the AR(1) model, the
Equation (2.53) is the one-step-ahead forecast oíyí+\. The one-step-ahead fore-
j-step-ahead forecast error is given by cast error is the difference between yí+x and E(yí+] so that et{\) — c/+1. To find the two-step-
et0') = ¿t+j + a\£t+j-\ + ar£t+j-2 + a]3£t+j-3 + • • • + a\hl£t+\ ( 2 - 5 ° ) Since the ahead forecast, update (2.52) by two periods
mean of (2.50) is zero, the forecasts are unbiased estimates of each valuéyr+j. The
v/+2 = a
0 + a
\yt+\ The
+ a
2); t + £
í+2 +
0\e í+\
proof is trivial. Since Eteí+j = Ete1+j^ = ... = Et£t+\ = 0> tne condi-tional expectation of ls
conditional expectation of y(+2
(2.50) is Ejejij) = 0. Since the expected valué of the forecast error is zero, the
E
forecasts are unbiased. ¡yt+2 = °o + °\Eiyt+\ + aiyt . (2.54)
Although unbiased, the forecasts from an ARMA model are necessarily inaecu- Equation (2.54) expresses the two-step-ahead forecast in terms of the one-step-
rate. To fmd the variance of the forecast error, continué to assume that the elements ahead forecast and current valué of yv Combining (2.53) and (2.54) yields
of the {s¡} sequence are independent with a variance equal to o¿. Henee, from (2.50) E ;
t) t+2= ao+ adao+ a\yt + a2yt~\ + P\£t\ + a2)'t
the variance of the forecast error is = £70(l + a x ) + [ax2 + a2]yt + «i^2.V/-i + ^\P\-i To find the
Var[<?r(/)] = <¿[\ + ci\2 + Q\A + a^ + ... + ¿j^O-D] (2.51)
Thus, the one-step-ahead forecast error variance is a2, the two-step-ahead fore- two-step-ahead forecast error, subtract (2.54) from yt+->. Thus,
cast error variance is ¿¿(l+aj2), and so forth. The essential point to note is that the er(2) = a, 0;/+1 ~ Ety,+,) + et+2 + A ^ l (2.55)
variance of the forecast error is an increasing function of/. As such, you can have Sincc>-/(_| - EtyH .¡ is the one-step-ahead forecast error, we can write the forecast
more confidence in short-term forecasts than in long-term forecasts. In the limit as error as
j —* oo, the forecast error variance converges to a2/{\ - É^2); henee, the forecast
error variance converges to the unconditional variance of the {_y,} sequence. <?,(2) = (ux + A)£-/+i +-5/+2 (2.56)
Moreover, assumingthe {e,} sequence is normally distributed. you can place con- I i n a l l y , all/-step-ahead forecasts can be obtained from
fidence intervals around the forecasts. The one-step-ahead forecast of.v,4l is a{) + tfjV,. £/.>'/+/ = a
0 + a E
\ tyt+j-\ + a
2 E t>'t+j-2*J ^ 2
(2-57)
82 CHAPTER 2 STATIONARYTIME-SERIES MODELS PROPERTIES OF FORECASTS 83

Equation (2.57) suggests that the forecasts will satisfy a second-order difference process given by (2.52). íf you could forecast the valué of y r +\ using (2.53), you
equation. As long as the characteristic roots of (2.57) lie inside the unit circle, the fore - vvoLild obtain the one-step-ahead forecast error
casts will converge to the unconditional mean: ao/() - a x - a2). We can use (2.57) to flnd e T (\) = y T+ ¡ -a o -a x y T -a 2 y T _ x - f\e T = e T+l
they-step-ahead forecast errors. Sinceyí+J- = o0 +axy1+j_\ + aiyt+j-2 + et+j + A^+y-b tne /-step-
ahead forecast error is Since the forecast error is the puré unforecasíable portion of yr+\> no other
ARMA model can provide you with superior forecasting performance. However, we
e,(j) = * iO'/+/-l ~ ^z+y-i) + ^2(V/+/-2 - E iy/+j-2> + £ /+j + /V/+/-1 need to estimate the parameters of the process, so our forecasts must be made using
= axet(J-\) + a2et(j-2) + et+J + Px eí+j_x (2.58) (2.59). As such, our forecast error will be
In practice, you will not know the actual order of the ARMA process or the actual e
valúes of the coefílcients of that process. Instead, to créate out-of-sample forecasts, it
r = y-r+1 - («o + a\y T + áiyr-i + AM
is necessary to use the estimated coefílcients from what you believe to be the most Clearly, the two forecast errors are not identical. When we forecast using (2.59),
appropriate forni of an ARMA model. The rule of thumb is that forecasts from an the coeíTicients (and the residuals) are estimated imprecisely. The forecasts made
ARMA model should never be trusted if the model is estimated with fewer than 50 using the estimated model extrapólate this coefficient uncertainty into the future.
observations. Suppose you have T observations of the {yt} sequence and choose to fít Since coefficient uncertainty increases as the model becomes more complex, it could
an ARMA(2, 1) model to the data. Let a hat or caret (Le.: A) over a parameter denote be that an estimated AR(1) model forecasts the process given by (2.52) better than an
the estimated valué of a parameter, and let {i,} denote the residuals of the estimated estimated ARMA(2, 1) model.
model. Henee, the estimated ARMA(2, 1) model can be written as How do you know which one of several reasonabie models has the best forecast-
ing performance? One way to answer these questions is to put the alternative models
y t = a 0 + aj^-i + a 2 y t _ 2 + £, +/M/-1
to a head-to-head test. Since the future valúes of the series are unknown, you can hold
Given that the sample contains T observations, the out-of-sample forecasts are back a portion of the observations from the estimation process. As such, you can esti-
easily constructed. For example, you can use (2.53) to forecast the valué of yT+í con- mate the alternative models over the shortened span of data and use these estimates
ditional on the T observations as to forecast the observations of the holdback period. You can then compare the prop-
E Tyr+\
= ¿ o + W T + ¿ 23 ? r-i + Á%- (2.59) erties of the forecast errors from the two models. To take a simple example, suppose
that {yt} contains a total of 150 observations and that you are unsure as to whether an
Once you know the valúes of ¿0,a,,á2> a°d ¡3l9 (2.59) can easily be constructed
AR(1) or an MA(1) model best captures the behavior of the series.
using the actual valúes yT, JY-l* and £T (i.e., the last residual of your estimated
One way to proceed is to use the fírst 100 observations to estimate both models
model). Similarly, the forecast ofyT+2 can be constructed as
and use each to forecast the valué ofjv^i oí - Since you know the actual valué of^jQi,
ETy T+2 = a o+aí ET y T+] + a2 y T you can construct the forecast error obtained from the AR(1) and from the MA(1).
These two forecast errors are precisely those that someone would have made if they
where ETyT+\ is the forecast from (2.59).
had been making a one-step-ahead forecast in period 100. Now, re-estimate anAR(l)
Given these two forecasts, all subsequent forecasts can be obtained from the dif-
and an MA(1) model using the fírst 101 observations. Although the estimated coeffi-
ference equation
cients will change somewhat, they are those that someone would have obtained in
ETyT+J = a 0 + a xETy T^j^ + a2 E ryr+ j-2 for
j> 2
period 101. Use the two models to forecast the valué of yXQ2. Given that you know
the actual valué of vl02, you can construct two more forecast errors. Since you know all
Unfortunately, it is much more difficult to construct confídence intervals for the
valúes of the {y{} sequence through period 150, you can continué this process so as
forecast errors. Not only is it necessary to include the effeets of the stochastic varia -
to obtain two series of one-step-ahead forecast errors, each containing 50 observations.
tion in the future valúes of {yr+/}> ** ls a*so necessary to incorpórate the fact that the
To keep the notation simple, let {fl¿} and {f2i} denote the sequence of forecasts from the
coefficients are estimated with error.
AR(1) and the MA(1), respectively. Similarly, {eu} and {e2¿} denote the sequences of
forecast errors from the AR(1) and the MA(1), respectively. If you understand the
Forecast Evaluation
notation, it should be clear that/jj = ^ioo3 ; lOl ls tne ^irst forecast using the AR(1), <?M
=>'ioi ~f\ \ is the fírst forecast error (where the fírst hold back observation Í S > ; J Q I ), and
Now that you have estimated a series and have forecasted its future valúes, the obvi-
ous question is, "How good are my forecasts?" Typically, there will be several plau- <?2,50 's tne ^ast forecast error from the MA(1).
sible models that you can select to use for your forecasts. Do not be fooled into Obviously, it is desirable that the forecast errors have a mean near zero and a
thinking that the one with the best flt is the one that will forecast the best. To make a small variance. A regression-based method to assess the forecasts is to use the 50
simple point, suppose you wanted to forecast the future valúes of the ARMA(2. 1) forecasts from the AR(1) to estimate an equation of the form
.Vioo+r = ^o + a\f\í + V \ Í
84 CHAPTER 2 STATIONARYTIME-SERIES MODELS PROPERTIES OF FORECASTS 85

If the forecasts are unbiased, an F-test should allow you to impose the restrictión Although it is common practice to assume that the {e¡} sequence is normally dis-
o0 = 0 and ax = 1. Similarly, the residual series {v,,} should act as a white-noise tributed, it is not necessarily the case that the forecast errors are normally distributed
process. It is a good idea to plot {vu} against {y]00+,} to determine if there are peri-ods with a mean valué of zero. Similarly, the forecasts may be serially correlated; this is
in which your forecasts are especially poor. Now repeat the process with the forecasts particularly true if you use multi-step-ahead forecasts. For example, equation (2.56)
from the MA(1). In particular, use the 50 forecasts from the MA(1) to estímate indicated that the two-step-ahead forecast error for y(+2 is
3'100+/ = b0 + b\f2i + V2/ / = 1, ..., 50 e?/ (2) = (c7, +^i )£•,+ ! + e l+2 and updating by one

Again, if you use an F-test, you should not be able to reject the joint hypothesis period yields the two-step-ahead forecast error for v/+3
bQ = 0 and bx = 1. If the significance levéis from the two F-tests are similar, you might e t+] (2) = (a l +ft)f /+2 +%3
select the model with the smallest residual variance; that is, select the AR(1) if
ít should be clear that the two forecast errors are correlated. In particular,
var(v1/)<var(v2r).
More generally, you might want to have a holdback period that differs from 50 E[et(2)et+l(2)]=(ax +Z?,)*2
observations. With a very small sample, it may not be possible to hold back 50 obser- The point is that predicting _y /+2 from the perspective of period / and predicting
vations. Small samples are a problem since Ashley (1997) shows that very large sam - v/+3 from the perspective of period /+T both contain an error due to the presence of
ples are often necessary to reveal a significant difference between the out -of-sample £t+2- However, for / > 1, E[e[(2)e(+¡(2)] = 0 since there are no overlapping forecasts.
forecasting performances of similar models. Henee, you need to have enough obser- Henee, the autocorrelations of the two-step-ahead forecast errors cut to zero after lag
vations to have well-estimated coefficients for the in-sample period and enough out-of- 1. You should be able to demónstrate the general result that /-step-ahead forecast
sample forecasts so that the test has good power. If you have a large sample, it is errors act as an MA(/-1) process.
typical to hold back as much as 50 percent of the data set. Also, you might want to use Finally, the forecast errors from the two alternative models will usually be highly
/-step-ahead forecasts instead of one-step-ahead forecasts. For example, if you . have correlated with each other. For example, a negative realization of £í+¡ will tend to
quarterly data and want to forecast one year into the future, you can perform the cause the forecasts from both modeis to be too high. Unfortunately, the violation of
analysis using four-step-ahead forecasts. Nevertheless, once you have the two any one of these assumptions means that the ratio of the MSPEs in (2.60) does not
sequences of forecast errors, you can compare their properties. have an F-distribution.
Instead of using a regression-based approach, many researchers would select the
model with the smallest mean square prediction error (MSPE). .If there are H obser- THE GBANGER-NEWBOLD TEST Granger and Newbold (1976) show how
vations in the holdback periods, the MSPE for the AR(1) can be calculated as to overeóme the problem of contemporaneously correlated forecast errors. Use the
i H two sequences of forecast errors to form
MSPE = ~ye2
x, = eu + e 2l and zf = eu - e2,
Given that the first two assumptions above are valid, under the nuil hypothesis
Several methods have been proposed to determine whether one MSPE is statis -
of equal forecast aecuracy, xt and zt should be uncorrelated
tically different from the other. If you put the larger of the two MSPEs in the numer-
Ex z
ator, a standard recommendation is to use the F-statistic Pxz = i r ^£(4 ~4t)
Model 1 has a larger MSPE if pxz is positive and model 2 has a larger MSPE if
^E^'E^í (2.60) PX2 is negative. Let rx= denote the sample correlation coefficient between {xt} and
1=1 /=1 { z ¡\ - Granger and Newbold (1976) show that
The intuition is that the valué of F will equal unity if the forecast errors from the rxz I]{\~r;z )l{H-\) (2.6 1)
two models are identical. A very large valué of F implies that the forecast errors from
has a /-distribution with //— 1 degrees of freedom. Thus, if rK7 is statistically di fíe r-
the flrst model are substantially larger than those from the second. Under the nuil
ent from zero, model 1 has a larger MSPE if r v, is positive and model 2 has a larger
hypothesis of equal forecasting performance, (2.60) has a standard F -distribution
MSPE if rvr is negative.
with (H, H) degrees of freedom if the following three assumptions hold:
1. The forecast errors have zero mean and are normally distributed THE DIEBQLD-lVJARiAISiG TEST There is a very large literature trying to
2. The forecast errors are serially uncorrelated extend the Granger-Ncwbold test so as to relax assumptions I and 2. Moreover,
aPplied econometricians might be interested in measures of forecasting perfomiance
3. The forecast errors are contemporaneously uncorrelated with each other
 86 CHAPTER 2 STATIONARYTIME-SERIES MODELS A MODEL OFTHE PRODUCER PRICE INDEX 87

othcr than the sum of squared errors. Indeed, it should be clear tbat using the sum of 10. A MODEL OFTHE PRODUCER PRICE INDEX
squared errors as a crilerion makes sense only if the loss from making an incorrect
forecast is quadratic. However, there are many other possibilities. For example, if The ARMA estimations performed in Section 7 were almost too straightforward. in
your loss depends on the size of the forecast error, you should be concerned with the practice, we rarely fmd a data series precisely conforming to a theoretieal ACF or
absolute valúes of the forecast errors. Alternatively, an options trader receives a pay- PACF. This section is intended to ¡Ilústrate some of the ambiguities frequently
off of zero if the valué of the underlying asset lies below the strike price but receives encountered in the Box-Jenkins technique. These ambiguities may lead two equally
a one-dollar payoff for each dollar the asset price rises above the strike price. Diebold skilled econometricians to estímate and forecast the same series using very different
and Mariano (1995) have developed a test that relaxes assumptions 1 to 3 and allows ARMA processes. Many view the necessity of relying on the researcher's judgment
for an objective function that is not quadratic. and experience as a serious weakness of a procedure that is designed to be scientific.
If we consider only one-step-ahead forecasts, we can elimínate the subscripty. As Yet, if you make reasonable choices, you will select models that come very cióse to
such, we can let the loss from a forecast error in period / be denoted by g(e¡). In the mimicking the actual data-generating process.
typical case of mean squared errors, the loss is ef. Nevertheless, to allow the loss It is useful to ¡Ilústrate the Box—Jenkins modeling procedure by estimating a
function to be general, we can write the dífTerential loss in period / from using quarterly model of the U.S. Producer Price índex (PPI). The data used in this section
model 1 versus model 2 as dt = g{e\¡) - g{e2d- Tne mean loss can be obtained as are the series labeled PPI on the file QUARTERLY.XLS. Exercise II at the end of
this chapter asks you to reproduce the results reported below.
^ = ÍE[^i/)-^2/)] (2.62) Panel (a) of Figure 2.5 clearly reveáis that there is little point in modeling the
series as being stationary; there is a decidedly positive trend or drift throughout the
Under the nuil hypothesis of equal forecast accuracy, the valué of d is zero. Since period l960gl to 200201. The first difference of the series seems to have a constant
d is the mean of the individual losses, under fairly weak conditions, the central limit mean, although inspection of Panel (b) suggests that the variance is an increasing
theorem implies that d should have a normal distribution. Henee, it is not necessary function of time. As shown in Panel (c), the first difference of the logarithm (denoted
to assume that the individual forecast errors are normally distributed. Thus, if we by Alppi) is the most likely candidate to be covariance stationary. Moreover, there is
knew var(¿7), we could construct the ratio d/jvar(d) and test the nuil hypothesis of a strong economic reason to be interested in the logarithmic change since Alppi t is a
equal forecast accuracy using a standard normal distribution. In practice, the imple- measure of inflation. However, the large volatility of the PPI accompanying the oil
mentation of the test is complicated by the fact that we need to estímate var(J). price shocks in the 1970s should make us somewhat wary of the assumption that the
If the {dj} series is serially uncorrelated with a sample variance of j0, the estí- process is covariance stationary. At this point, some researchers would make addi-
mate of var(J) is simply jo/(H-l). Since we use the estimated valué of the variance, tional transformations intended to reduce the volatility exhibited in the 1970s.
the expression d/^Jjo/{H — 1) has a /-distribution with H—\ degrees of freedom. However, it seems reasonable to estímate a model of the {Alppi t) sequence without
There is a very large literature on the best way to estímate the standard deviation any further transformations. As alvvays, you should maintain a healthy skepticism of
of d in the presence of serial correlation. Many of the technical details are not appro- the accuracy of your model.
priate here. Diebold and Mariano let y¡ denote the /-th autocovariance of the df Before reading on, you should examine the autocorrelation and partial autocorrela-
sequence. Suppose that the flrst q valúes of y¡ are different from zero. The variance tion functions of the {Alppi t) sequence shown in Figure 2.6. Try to identify the tentative
of d can be approximated by var(¿7) = [7() + 2yx + ... + 2jg](H— I)"1; the standard models that you would want to estímate. In making your decisión, note the following:
deviation is the square root. As such, Harvey, Leybourne, and Newbold (1998) rec- 1. The ACF and PACF converge to zero reasonably quickly. We do not want
ommend constructing the Diebold-Mariano (DM) statistic as
to overdifference the data and try to model the {A~lppit} sequence.
DM = d I ^(70+27, + ... + 2 7(? )/(//-!) (2.63) 2. The theoretieal ACF of a puré MA(<r/) process cuts off to zero at lag q and
the theoretieal ACF of an AR(1) model decays geometrically. Examination
Compare the sample valué of (2.63) to a f-statistic with H-\ degrees of freedom.6 of Figure 2.6 suggests that neither of these specifícations seems appropri
It is also possible to use the method for/-step-ahead forecasts. If {eu} and {e2t} ate for the sample data.
denote two sequences ofy-step-ahead forecasts, the DM statistic is
3. The ACF does not decay geometrically. The valué of px is 0.603 and the
DM - ^/^(7 ó +2 7 l +... + 2 -^) /[// + ] -2j + H - ] j(j- })} valúes of />>, p$, and p4 are 0.494, 0.451, and 0.446, respectively. Thus, the
ACF is suggestive of an AR(2) process or a process with both autoregres-
An example showing the appropriate use of the Granger-Newbold and sive and moving average components. The PACF is such that c^j = 0.604
Diebold—Mariano tests is provided in the next section. and cuts off to 0.203 abruptly (¡.e., éh2 = 0.203). Overall, the PACF sug
gests that we should consider models such that/? = I and p = 2.
 inexpensive, we can estímate a variety of models and compare their results. Tabie
2.4 reports estimates of six tentative models. To ensure comparabüity, all were esti-
mated over the same sample period. 7 Note the following points:
1. The estimated AR(l) model confirms our analysis conducted in the identi-
fication stage. Even though the estimated valué of a{ (0.603) is less than
unity in absolute valué and almost four standard deviations from zero, the
AR(1) specification is inadequate. Forming the Ljung~Box O-statistic for
4 lags of the residuals yields a valué of 13.9; we can reject the nuil that
Q(4) = 0 at the 1 percent signiflcance level. Henee, the lagged residuals of
this model exhibit substantial serial autocorrelation. As such, we must
elimínate this model from consideration.
2. The AR(2) model is an improvement over the AR(1) specification. The
estimated coefficients (o^ = 0.480 and a2 — 0.209) are each significantly
different from zero at conventional levéis and imply characteristic roots in
the unit circle. However. there is some ambiguity about the information
contení of the residuals. The O-statistics indícate that the autocorrelations
of the residuals are not statisticaliy significant at the 5 percent level but are
significant at the 10 percent level. As measured by the AÍC and SBC, the
flt of the AR(2) model is superior to that of the AR(1). Overall, the AR(2)
model dominates the AR(1) specification.
3. The ARMA(1, 1) specification is superior to the AR(2) model. The esti
mated coefficients are of high quality (with f-values of 14.9 and -4.41). The
4. Note the jump in ACF after lag 4 and the small jump in the PACF at lag 4 estimated valué of al is positive and less than unity and the O-statistics
(044 = 0.148 while <pS5 = -0.114). Since we are using quarterly data, we indícate that the autocorrelations of the residuals are not significant at con
might want to incorpórate a seasonal factor at lag 4. ventional levéis of statistical significance. Moreover, all goodness-of-fit
Points 1 to 4 suggest an ARMA(IJ) or an AR(2) model. In addition, we might measures select the ARMA( 1, 1) specification over the AR(2) model. Thus,
want to consider models with a seasonal term at lag 4. Since computing time is there is little reason to rnaintain the AR(2) specification.
90 CHAPTER 2 STATIONARYTIME-SERIES MODELS A MODEL OFTHE PRODUCER PRICE INDEX 91

Table 2.4 E stim ates of the PP I Having identified and estimated a plausible model, we want to perform additional
diagnostic checks of model adequacy. Due to the high volatility in the 1970s, the sam-ple
p=1 p=2 p=1 P =1 P=2 P=2
q=0 q=0 q=1 g=1,4 q=1 q=||4|| was split into the two subperiods: 1960g3 - 1971Q4 and 1972Q1 - 2002Q1 Model
estimates for each subperiod are
a 0 003 0.003 0.003 0.003 0.003 0.003
° (3.28) (2.48) (2.27) (2.31) (2.23) (2.87) A/pp/, = 0.002 + 0.621 A/pp/,_, + st - 0.329£>_, + 0.263 ¿r,^ (1960g3 - 1971Q4) and
a 0 603 0.480 0.871 0.768 0.949 0.460
1
(9.69) (6.24) (14.9) (9.46) (3.97) (5.87) A/pp/, = 0.002 + 0.763A/pp/,_i + e, - 0.350^, + 0.301 £-,.4 (1972Q1 - 200201)
0.209 -0.061 0.163
The coefficients of the two models appear to be quite similar; we can formally test
2
(2.65) (-0.348) (2.03)
for the equality of coefficients using (2.47). Respectively, the sum of squared
/?, -0.455 -0.358 -0.517
(-4.41) (-3.41) (-2.33)
residuals for the two models are SSR! = 0.001267 and SSR 2 = 0.017551.
0A 0.297 0.261 Estimating the model over the full sample period yields SSR = 0.018870. Since T
(3.63) (3.23) — 167 and n = 4 (including the intercept means that there are four estimated coeffi-
SSR 0.0210 0.0201 0.0196 0.0188 0.0196 0.0191 cients), (2.47) becomes
AIC -637.2 -642.3 -646.5 -651.4 -644.7 -648.3
SBC -631.0 -632.9 -637.2 -639.0 -632.3 -635.9
F= [(0.018870 - 0.001267 - 0.017551)/4] / [(0.001267 + 0.017551)/(167 - 8)] =
Q(4) 13.9 (0.008) 8.5 (0.074) 4.5 (0.339) 1.4 (0.846) 4.3 (0.367) 2.1 (0.720)
0.10984
Q(8) 21.9 (0.005) 15.3 (0.053) 11.5 (0.176) 7.3 (0.505) 11.4 (0.181) 8.3 (0.401) With 4 degrees of freedom in the numerator and 159 in the denominator, we can-
Q(12) 27.2 (0.007) 20.7 (0.054) 18.0 (0.104) 11.2 (0.513) 18.7 (0.096) 12.2 (0.426) not reject the nuil of no structural change in the coefficients (i.e., we accept the
Notes: hypothesis that there is no change in the structural coefficients).
Each coefficient is reported with the associated í-statistic for the nuil hypothesis that the
estlmated valué ¡s equal to zero. Out-of-SampIe Forecasts
SSR is the sum of squared residuals.
Q{n) reports the Ljung-Box Ü-statistic for the autocorrelations of the n residuals of the estimated
model. Significance levéis are in parentheses.
We can assess the forecasting performance of the ARMA(1, 1) and ARMA(1, (1, 4)) by
||4||denotes that only the fourth MA term was ¡ncluded in the estimated model. using the Granger—Newbold and Diebold-Mariano tests discussed in the previous section.
Given that the data set contains a total of 167 usable observations, it is possi-blc to use a
holdback period of 50 observations. This way, there are at least 117 observations in each
4. In order to account for the possibility of seasonality, we estimated the of the estimated models and an adequate number of out-of-sample forecasts. First, the
ARMA(1, 1) model with an additional moving average coefficient at lag two models were estimated using aü available observations through 1989Q3 and two
4—that is, we estimated a model of the form yt ~ o0 + tfiV,_i + et + /3\£t_\ one-step-ahead forecasts were obtained. The actual valué of A/pp/1989:4 = 0.00385; the
+ P^t-A- Other seasonal patterns are considered in the next sec-tion. For ARMA(1, 1) predicted a valué of 0.00715 and the ARMA(1, (1, 4)) model predicted a
now, note that the additive expression /? 4£>_^ is often preferable to an valué of 0.00415. Thus, the forecast of the ARMA(1, (1,4)) is superior to that of the
additive autoregressive term of the form aAyf_^. For truly seasonal ARMA(1, 1) for this first period. An additional 49 forecasts were obtained for periods
shocks, the expression 0^et^ best captures spikes—not decay—at the 1990gl to 2002gl. Let eu denote the forecast errors from the ARMA(1, 1) model and e2t
quarterly lags. The slope coefficients of the estimated ARMA(1, (1,4)) denote the forecast errors from the ARMA(1, (1, 4)) model. The mean of eu is -
model are all highly significant with r-statistics of 9.46, -3.41, and 3.63.8 0.00210, the mean of e2í is -0.002250, and the estimated variances are such that
The Q-statistics of the residuals are all very low, implying that the auto- var(é>lr) = 0.000127 and var(e2/) = 0.000133. As such, there is an advantage in the
correlations are not statistically different from zero. Moreover, the AIC forecasting performance of the ARMA(1, 1). To ascertain whether this difference is
and SBC select this model over the ARMA(1, 1) model. statistically significant, we first use the Granger-Newbold test. Form the x¡ and zt series
5. In contrast, the ARMA(2, 1) contains a superfluous coefficient. The f-statistic as xt — t>\t + e2t and z{ ~ eu - e2l. The correlation coefficient between xt and zv is rx= =
for a2 is suíficiently low that we should elimínate this model. -0.0767. ■ Given that there are 50 observations in the holdback period, form the
6. Since a seasonal term seems to fit the data well, we estimated añ equation Granger-Newbold statistic
of the form v, = a0 + axyt^ + #2>'/-2 + £i + l%ei-A- Notice that all coeffi r x z /yJ(\-r; z )/(H-\) = -0.07677^/(1-(-0.0767) 2 /49 = -0.5387
cients have í-values in excess of 2.0 and that the £)-statistics are not signif
icant. However, as measured by the AIC and the SBC, this model does not
fit the data as well as the ARMA(1, (1; 4)).
 92 CHAPTER 2 STAT1ONARYTIME-SERIES MODELS SEASONALITY 93

With 49 degrees of freedom, a valué of / = -0.5387 is nol statistically signiílcant. ARMA(K 1) and ARMA(1, (1 , 4)) models yield estimated valúes of a,
We can conclude that the forecasting performance of the ARMA(1, 1) is not statisti- (0.871 and 0.768, respectively), vvhich are cióse to unity. Some researchers
cally different from that of the ARMA(1, (1, 4)). might have chosen to model the second-order difference of the series. Others
We obtain virtually the same answer using the DM statistic. Oftentimes forecast- might have detrended the data using a deterministic time trend.
ers are concerned about the MSPE. However, there are many other possibilities. In 2. Seasonality: The seasonality of the data was modeled using a moving
Exercise 11 at the end of this chapter, you will be asked to use the mean absolute average term at lag 4. However, there are many other plausible ways to
error. Now, to illustrate the use of the DM test, suppose that the cost of a forecast error model the seasonality in the data, as will be discussed in the next section.
rises extremely quickly in the size of the error. In such circumstances, the loss func- For example, many computer programs are capable of estimating multi-
tion might be best represented by the forecast error raised to the fourth power. Henee, plicative seasonal coefficients. Consider the multiplicative seasonal model
dt ^eu-e2í (1 - a xL)yt = (1 + 0 X L){\ + 0 4L 4 )£ (

The mean valué of the {d}} sequence (i.e., d) is —2.995 ■ 10~9 and the estimated Here the seasonal expression flA£t_^ enters the model in a multiplicative,
variance is 4.6487 • 10~16. Since H =■ 50, we can fomi the DM-statistic rather than a linear, fashion. Experimenting with various multiplicative
seasonal coefficients might be a way to improve forecasting performance.
DM = -2.995 • 10~9 / (4.6487 • 1(H6 / 49)1/2 = -0.972
3. VolatHity: Given the volatility of the {Alppif} sequence during the 1970s, the
If there were serial correlation in the {cft} series, we would need to use the spec- assumption of a constant variance might not be appropriate. Transforming the
ifícation in (2.62). Toward this end, we would select the statistically significant val- data using a square root, rather than the logarithm, might be more appropri
úes of jq. However, in this exaniple, we do not need to worry about serial correlation ate. A general class of transformations was proposed by Box and Cox (1964).
in the {át} sequence because the Ljung-Box statistics do not indicate that the auto- Suppose that all valúes of \yt} are positive so that it is possible to construct
correlations are significant. The individual autocorrelations are the transformed [y*) sequence as
P\ Pl Pl PA P5 P6 Pl P% P9 P]Q P\\ P\2
-0.18 0.06 -0.17 -0.10 0.07 0.12 0.13 -0.05 0.08 0.00 0.12 0.12 = ln(y,) ^=°
Instead, suppose we used the squared forecast errors as a measure of the loss so The common practice is to transform the data using a preselected valué of
that dt = e\t - e\t. Now 2 = -5.863 • 10"6 and the estimated variance is 1.8703 -lO"9. If Á. Selecting a valué of A that is cióse to zero acts to smooth the sequence.
you check the autocorrelations, you will fínd the following: As in the PPI example, an ARMA model can be fitted to the transformed
data. Although some software programs have the capacity to simultane-
P\ Pl P3 P4 PS P6 Pl P% P9 P]Q P\\ P\2
ously estímate A along with the other parameters of the ARMA model, this
0.08 -0.09 -0.23 -0.30 0.32 0.16 0.29 0.13 -0.27 -0.06 0.08 0.22 approach has fallen out of fashion. instead, it is possible to actually model
The Ljung-Box Q(4) is not significant at the 5 percent level, but the Q(8) has a the variance using the methods discussed in Chapter 3.
pro¿>-value of 0.004. If we construct the DM statistic using all 12 autocovariances
(recall p¡ = 7,-/ 70), we fínd that 70 + 2y¡ + ... + 2jl2 = 5.957 • 10~9. Thus, the desired 11. SEASQ1MAL1TY _______________________________________
*> valué of the DM statistic is Many economic processes exhibit some form of seasonality. The agricultural, con-
DM =-5.863 -lO"6/(5.957 • 10~9 / 49)1/2 = -0.532 struction, and travel sectors have obvious seasonal patterns resulting from their
If we now compare this valué to a /-statistic with 49 degrees of freedom, we conr dependence on the weather. Similarly, the Thanksgiving-to-Christmas holiday season
elude that the difTerence between the forecasting abilities of the two models is not sta- has a pronounced influence on the retail trade. In fact, the seasonal variation of a
tistically significant. This is not too surprising since Ashley (1997) shows that very series may account for the preponderance of its total variance. Forecasts that ignore
large samples are often necessary to reveal a significant difference between the out- important seasonal patterns will have a high variance. In the last section, we saw how
of-sample forecasting performances of similar models. the inclusión of a four-quarter seasonal factor could help improve the model of the
Although the ARMA(1, 1) and ARMA(1, (1, 4)) appear to be adequate, other PPI. This section expands tru t discussion by illustrating some of the techniques that
researchers might have selected a decidedly different model. Consider some of the can be used to identify seasonal patterns.
alternatives: Too many people i all into ihe trap of ignoring seasonality if they are working with
(leseasonalized or seasonally adjusted data. Suppose you collect a data set that the
1. Trends: Although the logarithmic change of the PPI appears to be station- U.S. Census Burean has "seasonally adjusted" using its X-ll method.9 In principie.
ary. the ACF converges to zero rather slowly. Moreover, both the
94 CHAPTER 2 STATI0NARYTIME-SER1ES MODELS
SEASONALITY 95

trie seasonally adjusted data should liave the seasonal pattern removed. However, cau- Equation (2.67) differs from (2.66) in that it allows the moving average term at
tion is necessary. Although a standardized procedure may be necessary for a govern- iag 1 to interact with the seasonal moving average effect at lag 4. In the same wav
ment agency reporting hundreds of series, the procedure might not be best for an (2.68) allows the autoregressive term at lag 1 to interact with the seasonal autoregres-
individual wanting to model a single series. Even if you use seasonally adjusted data, sive effect at lag 4. Many researchers prefer the multiplicative form since a ríen inter-
a seasonal pattern might remain. This is particularly true if you do not use the entire action pattern can be captured with a small number of coefficients. Rewrite (2.67) as
span of data; the portion of the data used in your study can display more (or less) sea- yt = a\y¡-\ +
£> +
ft\£t-\ +
04£t-4 +
Pxfat-S
sonality than the overall span. In fact, the PPI sequence used in the previous section
Estimating only three coefficients (i.e., c/h /?j, and /34) allows us to capture the
had been deseasonalized by the Bureau of Economic Analysis. There is another impor-
effects of an autoregressive term and the effects of moving average terms at lags 1, 4,
tant reason to be concerned about seasonal i ty when using deseasonalized data. Implicit
and 5. Of course, you do not really get something for nothing. The estimates of the
in any method of seasonal adjustment is a two-step procedure. First, the seasonality is
three moving average coeffícients are interrelated. A researcher estimating the uncon-
removed and second, the autoregressive and moving average coefficients are estimated
strained model y¡ — a{yt_x + s¡ + /?|£>_j + P^t-A + 05£t-5 would necessarily obtain a
using Box—Jenkins techniques. As surveyed in Bell and Hillmer (1984), oftentimes the
smaller residual sum of squares. However, (2.67) is clearly the more parsimonious
seasonal and the ARMA coeñicients are best identifíed and estimated jointly. In such
model. If the unconstrained valué of/i5 approximates the product jS^j3A, the multi-
circumstances, \i is wise to avoid using seasonally adjusted data.
plicative model will be preferable. For this reason, most software packages have rou-
tines capable of estimating multiplicative models. Otherwise, there are no theoretical
Models of Seasonal Data
grounds leading us to prefer one form of seasonality over another. As illustrated in the
The Box—Jenkins technique for modeling seasonal data is only a bit different from last section, experimentation and diagnostic checks are probably the best way to
that of nonseasonal data. The twist introduced by seasonal data of period s is that the obtain the most appropriate model.
seasonal coefficients of the ACF and PACF appear at lags s, 2s, 3s, ..., rather than at
lags 1, 2, 3, .... For example, two purely seasonal models for quarterly data might be Seasonal Differencing
y, = a^^ + et (2.64) The Christmas shopping season is accompanied by an unusually large number of trans-
and actions, and the Federal Reserve expands the money supply to accommodate the
yt = et + PAet_4 (2.65)
increased demand for money. As shown by the dashed line in Figure 2.7, the U.S. money
supply, as measured by MI, has a decidedly upward trend. The series, called M1NSA,
You can easily convince yourself that the theoretical correlogram for (2.64) is is contained on the file QUARTERLY.XLS. You can use the data to follow along with
such that: p¡ — {a¿)il4 if i/4 is an integer and p¡ — 0 otherwise; thus, the ACF exhibits
decay at lags 4, 8, 12, .... For model (2.65), the ACF exhibits a single spike at lag 4
and all other correlations are zero.
In practice, identifícation will be complicated by the fact that the seasonal pat-
tern will interact with the nonseasonal pattern in the data. The ACF and PACF for a
combined seasonal/nonseasonal process will reflect both elements. Note that the final
model of the PPI estimated in the last section had the form
yt = a\>?t-\ + e, + P\ £/-i + PA^-4 ;, , (2-66)
Alternatively, an autoregressive coefficient at lag 4 might have been used to cap-
ture the seasonality
= a + + £ +
yt \yt-\ *4 >'/-4 t 0\ ¿t-\
Both of these methods treat the seasonal coefficients additively; an AR or an
MA coefficient is added at the seasonal period. Multiplicative seasonality allows
for the interaction of the ARMA and the seasonal effects. Consider the multiplica-
tive specifications
(\'a}L)y, = (1 + fyL)(\+h%L*)st (2.67)

(\-axL){\-aAL*)yx = (1 + 0xL)£t (2.68)

96 CHAPTER 2 STATIONARYTiME-SERIES MODELS
Thus, we use the seasonal differenee of the fírst difference. The ACF and PACF
the discussion below. The logarilhinic change, shown by the solid line, appears lo be sta- for the {/??,} sequence are shown in Panel (/?) of Figure 2.8; íhe properties of this
tionary. Nevertheless, there is a clear seasonal pattem in that the valué of the fourthquar- series are much more amenable to the Box—Jenkins methodology. The autocorrelation
ter for any year is substantially higher than that for the adjacent quarters. and partía! autocorrelations for the fírst lew lags are strongly suggestive of an AR(1)
This combination. of strong seasonality and nonstationarity is often found in eco- process (/;, = ou =0.38,/>> = 0.16, and <^2 = 0.02). Recall that the ACF foran AR(1)
nomic data. The ACF for a process with strong seasonality is similar to that for a non- process will decay and íhe PACF will cut to zero after lag 1. Given that p4 = —0.28,
seasonal proeess; the main difference is that the spikes at lags s, 2s, 3s, ... do not exhibit p5 = 0.01, é44 = -0.34, and c655 = 0.28, there is evidence of remaining seasonality in
rapid decay. We know that it is necessary to difference (or take the logarithmic change the { i n f } sequence. The seasonal term is most likely to be in the form of an MA coef-
of) a nonstationary process. Similarly, if the autocorrelations atthe seasonal lags do not ficient since the autocorrelation cuts to zero whereas the PACF does not.
decay, it is necessary to take the seasonal difTerence so that the other autocorrelations Nevertheless, it is best to estímate several similar models and then select the best.
are not dwarfed by the seasonal eñects. The ACF and PACF for the growth rate of MI Esümates of the following three models are reported in Table 2.5:
are shown in Panel (a) of Figure 2.8. For now, just focus on the autocorrelations at the mt = ü0 + Í/J/77,_I + ei + /V4-/-4 Model 1: AR(1) with Seasonal MA
seasonal lags. All seasonal autocorrelations are large and show no tendency to decay. In m( — ÜQ + (1 + fl|/-)(l + a4L^)m¡_\ + st Model 2: Multiplicative Autoregressive
particular, p4 = 0.66, p% = 0.52, pn = 0.43, pl6 = 0.42, p^ = 0.47, and pj4 = 0.49. These m, = a0 + (1 + ,/^L)(l + ft4LA)£t Model 3: Multiplicative Moving Average
autocorrelations reflect the fact that the change in MI from one Christmas season to the The point estimates of the coefficients all imply stationarity and invertibility.
next is not as pronounced as the change between the fourth quarter and other quarters. Moreover, all are at lcast six standard deviations from zero. However, the diagnostic sta-
The fírst step in the Box—Jenkins method is to transform the data so as to make it tistics all suggest that model 1 is preferred. Model 1 has the best fit in that it has the íow-
stationary. As such, a logarithmic transformation is helpful because it can straighten cst sum of squared residuals (SSR). Moreover, the Q-statistics for lags 4, 8, and 12
the nonlinear trend in MI. Let yt denote the log of MI. As mentioned above, the fírst indicate that the residual autocorrelations are insignificant. In contrast, the residual cor-
difference of the {y,} sequence, illustrated by the solid line in Figure 2.7, appears to relations for model 2 are signiñeant at the long lags [i.e., Q(S) and Q(12) are significant
be stationary. However, to remove the strong seasonal persistence in the data, we need at the 0.007 and 0.002 levéis]. This is because the multiplicative seasonal autoregressive
to take the seasonal diflference. For quarterly data, the seasonal difference isj^ — j'/_4- (SAR) term does not adequately capture the seasonal pattem. An SAR term implies
Since the order of differencing is irrelevant, we can form the transformed sequence autoregressive decay from period s into period .v + 1. ín Panel (b) of Figure 2.8, the valué
m^O-LXl-L^,

Tabíe 2.5 Three Models of Money Growth

Model 1 Model 2 Model 3

a-, 0.543 0.492

(7.98) (6.98)
a4 -0.437
(-5.81)
¿T 0.460
(6.30)
.-í4 -0.763 -0.752
(-14.27) (-13.88)
SSR 0.0161 0.0196 0.0175
AIC -6 5 0. 7 -619.4 -637.0
SBC -641.5 -610.2 -627.8
Q(4) 2.03 (0.680) 3.63 (0.458) 21 35 (0.000)
Q(8) 6.41 (0.601) 22.20 (0.007) 28.37 (0.000)
Q(12) 14.11 (0.294) 30.89 (0.002) 40.04 (0.000)
To ensure comparabüiíy, íhe three models are estimated
over the 1963O3-2002Ü1 period. The estimated intercepts
are not reported since all were insignificant from zero.
 ;
SUMMARY AND CONCLUSÍONS 90
98 CHAPTER 2 STATIONARYTIME-SERIES MODELS

A compact notation has been developed that allows for the effícient representa-
of p4 is -0.28 but p5 is almost zero. As such, a multiplicative seasonal moving average tion of intricate models. As in previous sections, the chh difference of a series is
(SMA) temí is more appropriate. Model 3 properly captures the seasonal pattern, but the denoted by Ar/. Menee,
MA(1) temí does not capture the autoregressive decay present at the short lags. Other
diagnostic methods, including splitting the sample, suggest that model 1 is appropriate. A^^AO^-vV-i)
The out-of-sample forecasts are shown in Figure 2.9. To créate the one- through
twelve-step-ahead forecasts, model 1 was estimated over the full sample period A seasonal difference is denoted by As where s is the period of the data. The
196ÍQ3- 200221 • The estimated model is D-th such seasonal difference is Af. For example, if we want the second seasonal
m, = 0.529m/_1 + et - 0.758¿r,_4 (2.69) difference of a monthly series, we can form
Given that w2Oo2:l = 0.00795 and the residual for 2001:02 was 0.0119 (i.e.,
£200122 = 0.0119), the forecast of m2oo2:2 ^s -0-00490. Now, use this forecast and the = A 12>V - A I2> ? /-12
valué of £200103 to forecast w?2002<23' You can continué in this fashion so as to obtain = 3 '/ ~y t - \ 2 - (> V - 1 2 - > 7- 2 4 )
the out-of-sample forecasts for the {mt} sequence. Although you do not have the
residuals for periods beyond 2002gl, you can simply use their forecasted valúes of = yt-2yt-\2+y.t-24
zero. The trick to forecasting future valúes of MI from the {mt} sequence is to sum Combining the two types of differencing yields A^Af. Multiplicative models are
the changes and the seasonal changes so as to obtain the logarithm of the forecasted written in the form ARlMA(p, d9 q)(P, D9 Q)5 where: p and q = the nonseasonal
valúes of MI. Since mf = (1 - L){\ - L4)ln(M 1,), it follows that the valué of ln(Ml,) ARMA coefficients
can be obtained from mt + lnCMl^j) + ln(Ml,_4) - ln(Ml,__5). The fírst 12 of the fore- d = number of nonseasonai differences
casted valúes are plotted in Figure 2.9. P = number of multiplicative autoregressive coefficients
The procedures illustrated in this example of fitting a model to highly seasonal D = number of seasonal differences
data are typical of many other seríes. With highly seasonal data it is necessary to sup-
Q = number of multiplicative moving average coefficients
plement the Box-Jenkins method:
s = seasonal period
1. In the identificaron stage, it is usually necessary to seasonally difference Using this notation, we can say that the fitted model of PPI is an ARIMA(1, 1,
the data and to check the ACF of the resultant series. Oftentimes, the sea 0)(0, 1, 1)4 model. Moreover, the valué of mt can be written as m¡ = AA4 In(Ml,). In
sonally differenced data will not be stationary. In such instances, the data
applied work, the AR1MA(1, 1, 0)(0, 1, \) s and the ARIMA(0, 1, l)(0, 1, l)5 models
may also need to be first differenced.
oceurs routinely; the latter is called the airline model ever since Box and Jenkins
2. Use the ACF and PACF to identify potential models. Try to estímate mod- (1976) used this model to analyze airline travel data.
els with low-order nonseasonal ARMA coefficients. Consider both additive
and multiplicative seasonality. Allow the appropriate form of seasonality to 12. SUMMARY AND CONCLUSÍONS ________________
be determined by the various diagnostic statistics.
The chapter focuses on the Box-Jenkins (1976) approach to identification, estimation, diagnos
tic checking, and forecasting a univariate time-series. ARMA models can be viewed as a spe-
cial class of linear stochastic difference equations. By definition, an ARMA model is
covariance stationary in that it has a finite and time-invaríant mean and covariances. For an
ARMA model to be stationary, the characteristic roots of the difference equation must lie inside
the unit circle. Moreover, the process must have started infinue.ly far in the past or the process
must alvvays be in equilibrium. .•/■*
In the identification stage, the series is plotted and the sample autocorrelations and partial
correlations are examinad. As ill ustrated using the U.S. PPI, a slowly decaying autocorrelation
function suggests nonstationarity behavior. In such circumstances, Box and Jenkins recommend
differencing the data. Formal tests íbr nonstationarity are presented in Chapter 4. A common
practice is to use a logarithmic or Box -Cox trahsformation if the variance does not appear to be
constant. Chapter 3 presents some modern techniques that can be used to model the variance.
The sample autocorrelations and partial correlations of the.suitably transformed data are
compared to those of various theoretical ARMA processes. All plausible models are esti-
mated and compared using a battery of diagnostic criteria. A well-estimated model: (i) is par-
simonious: (ii ) has coefficients that imply s'tationarity and invertibility; (iii) fus the data
 QUESTIONS AND EXERCISES 101
100 CHAPTER 2 STATIONARYTIME-SERIES MODELS

6. Consider the second-order stochastic dilTcrcnce equation: y, = 1.5vy , - Q.5y,_ 2 + e t .

well; (iv) has residuals that approximate a white -noise process; (v) has coefficienls that do a. Find the characteristic roots of the homogeneous equation.
not change over the sample period; and (vi) has good out -of-sample forecasts.
b. Demónstrate that the roots of I - 1.5L + 0.5L 2 are the reciprocáis of your answer
In utilizing the Box—Jenkins methodology, you will find yourself making many secmingly
adhoc choices. The most parsimonious model may not have the best fit, but may have the best in part a.
out-of-sample forecasts. You will ílnd yourself addressing the following types of questions: c. Given initial conditions for y {) and V|, f\nú the solution for y, in terms of the current
What is the most appropriate data transformation? Is an ARMA(2, 1) model more appropriate and past valúes of the {£,) sequence. Explain why it is not possible to obtain the
than an ARMA(1, 2) specificatión? How can seasonality best be modeled? Given this latitude, backward looking solution for y, unless such initial conditions are given.
many view the Box-Jenkins methodology as an art rather than a science. Nevertheless, the d. Find the forecast function for VY+.V 0 e - * ^inc^ lne solution for any valué of yT+s
technique is best learned through experience. The exercises at the end of this chapter are given the valúes ofy T and Vj_\).
designed to guide you through the types of choices you will encounter in your own research. e. Find £>,, Eyl+U varO-'/), varCv^j), and cov0',+ h. v/)-
7. The file entitled SlM_2.XLS contains the simuiated data sets used in this chapter. The
QUESTIONS AND EXERCISES
first series, denoted Y I, contains the 100 valúes of the simuiated AR(l) process used in
1. In the coin-tossing example of Section 1, your average winnings on the last four tosses Section 7. Use this series to perform the following tasks (Note: Duc to differences in
(w t ) can be denoted by data handling and rounding, your answers need only approximate those presented here):
w, = \/4£ { + \/4e,_ } + l/4^_ 2 + l/4¿r,_ 3 a. Plot the sequence against time. Does the series appear to be stationary?
a. Find the expected valué of your winnings. Find the expected valué given that e t _ 3 b. Verify that the first 12 coefficients of the ACF and the PACF are as follows:
= e>.2 = ] • L ACF
b. Find var(w,). Find var(w f) conditional on e t_3 = e t_2 ~ 1 ■ 1: 0.7394472 0.5842742 0.4711050 0.3885974 0.3443779 0.3350913
c. Find cov(w/5 *>,_]), cov(vt>,, w,_2) and eov(w,, w,_s). 7: 0.2972263 0.3251532 0.2689484 0.2007989 0.1886648 0.0824283
2. Substitute (2.10) into y t = a 0 + tfjJV-1 + e t . Show that the resulting equation is an identity. PACF
a. Find the homogeneous solution to y t ~ a 0 + a^y^ + e t . 1: 0.7394472 0.0827240 0.0302925 0.0255945 0.0601115 0.0889358
b. Find the particular solution given that \ a x | < 1. 7: -0.0165339 0.1438633 -0.1002335 -0.0653566 0.0699036 -0.2040202
c. Show how to obtain (2.10) by combining the homogeneous and particular solutions. Ljung-Box 2-Statistics: 0(8) = 177.58, g(16) = 197.84, O(24) - 201.28.
3. Consider the second-order autoregressive process y t - a Q + a 2 y t _2 + ¿i where \a 2\ < 1. c. Use the data to verify the results given in Table 2.2.
a. Find: i. E^y, ii. £,_1>v iii. Ety1+2 iv. Cov^y^)
d. Estímate the series as an AR(2) process without an intercept. You should obtain:
v. Cov(yf, yt_2) vi. the partial autocorrelations <p } j and (fo
b. Find the impulse response function. Given y t _ 2, trace out the effects on an e t shock Included observations: 98
on the lVf} sequence. Cocflicicnt Estímate Standard Error r-Statistic ProbabiHty
c. Determine the forecast function: E f y t+S . The forecast error e,(s) is the difference
a, 0.701 0.102 6.878 0.0000
between.y/+5 and E t y t +S . Derive the correlogram of the (e,($)} sequence. {Hint:
a2 0.105 ________ 0.101___________1.036 0.3030
Find E t e r(s), var[e,(-s)], and E ([e f (s)e f (s-j)] for y = 0 to $}.
Ljung-Box 0-Stalistics: 0(8) = 5.1317. O(16)= 15.8647. Q(24) = 21.0213
4. Two different balls are drawn from a jar containing three balls numbered 1, 2, and 4.
Let x denote number on the first ball drawn and y denote sum of the two balls drawn. c. Estímate the series as an ARMA(1, 1) process without an intercept. You should obtain:
a. Find the joint probability distribution for x and y; that is, find prob(x = 1, y = 3), Included observations: 99 Coefficient Estímate Standard Error / -Statistic
prob(x = 1, y = 5). ..., prob (x = 4, y =6). Probability
b. Find each of the following: E(x), E(y\ £(y JC = 1), E(x\y = 5), var(*ly = 5), and'£(y2).
c. Consider the two functions w } = 3x 2 and M -, = x ~] . Find E(w } + w n ) and £( M '¡ + a{ Ü.H46 0.069 12.248 0.0000
M-'2|V=3). " " ■
.y, -0.147 0.126 -1.162 0.2481

d. How would your answers change if the balls were drawn with replacement? Verify that the ACF' and PACF of the residuals do not indícate any serial corrclation.
8. The second cokunn in file SIM_2.XLS contains the 100 valúes of the simuiated
5. The general solution to an wth-order difference equation requires n arbitrary constants.
ARMA(1, 1) process used in Section 7. This series is entitled Y2. Use this series to per
Consider the second-order equation y, = a 0 + 0.75v,_, - 0.125v,_-> + s r
form the following tasks {Xote: Due to differences in data handling and rounding, your
a. Find the homogeneous and particular solutions. Discuss the shape of the impulse
response function. answers necá only approximate those presented here):
a. Plot the sequence against time. Does the series appear to be stationary? Verify that
b. Find the valúes of the initial conditions that ensure the {y,} sequence is stationary.
the first 12 coefficients of the ACF and the PACP" are as follows:
c. Given your answer to pan b. derive the correlogram for the {v,} sequence.
102 CHAPTER 2 STATIONARYTIME-SERIES MODELS QUESTIONS AND EXERCISES 103

ACF a. Show that all three methods ofcalculating the AIC will necessarily select the same
1: -0.8343833 0.5965289 -0.4399659 0.3497724 -0.3187446 0.3316348 model. [¡lint: The AIC will select model I over model 2 if the AIC from model I
7: -0.3371782 0.3166057 -0.2761498 0.1789268 -0.0839171 0.0375968 is smaller than that from model 2. Show that all three ways ofcalculating the AIC
PACF are monotonic transformations of each other.]
1:-0.8343833 -0.3280611 -0.1942907 -0.0145160 -0.1398293 0.0891764 b. Show that all three methods ofcalculating the SBC will necessarily select the
7: 0.0004335 0.0143663 0.0166776 -0.1987829 -0.0462213 -0.0212410 same model.
b. Verify the results in Table 2.3. c. Select one of the three pairs above. Show that the AIC will never select a more par-
c. Estímate the process using a puré MA(2) model. You should obtain:
simonious model than the SBC.
Included observations: 100 11. The fileQUARTERLY.XLS contains U.S. PP1 data from 196O:01to 2002:01. Make the
Coefficient Estímate Standard Error / -Statistic Probabüiry data transformations indicated in Section 10 of the text.
a. Use the full sample period to obtain the estimates of the ARMA(1, 1) and
0x -1.191 0.068 -17.41 0.0000
ARMA(1, (1,4)) and AR(2) model reported in Section 10.
/?> _________ 0.468 _________ 0.081 ______ 5.77 0.0000
Verify that the Ljung—Box 0-Statistics a re : b. Estimate the ARMA(1, 1) and ARMA(1, ( 1 , 4 ) ) models over the period 1960:01
to 1989:03. Obtain the one-step-ahead forecast and the one-step-ahead forecast
0(8) = 28.4771. Signifícance level 0.00007638
error from each. As in Section 10, continué to update the estimation period so as
0(16)= 37.4666. Signifícance level 0.00062675
to obtain the fífty one-step-ahead forecast errors from each model. Let {eu}
g(24) - 38.8424. Signifícance level 0.01470990
denote the forecast errors from the ARMA(I, 1) and {e2í} denote the forecast
d. Compare the MA(2) to the ARMA(1, 1). errors from the ARMA(1, (1, 4)). You should find that the properties of the fore
9. The third column in file S1M_2.XLS contains the 100 valúes of the simulated AR(2) cast errors are such that
process used in Section 7. This series is entitled Y3. Use this seríes to perform the fol - Series Obs Mean Standard Error Mínimum Máximum
lowing tasks {Note: Due to differences in data handling and rounding, your answers
need only approximate those presented here): eu 50 -0.00210 0.011283 -0.03892 0.01861
a. Plot the sequence against time. Verify the ACF and the PACF coefficients reported ev 50 -0.00225 0.011516 _____ -0.03894 0.02006
in Section 7. Compare the sample ACF and PACF to the those of a theoretical c. Construct the Diebold-Mariano test using the mean absolute error. How do the
AR(2) process. results compare to those reported in Section 10?
b. Estímate the series as an AR(1) process. You should fínd that the estimated AR(1) d. Use the Granger-Newbold test to compare the AR(2) model to the ARMA(1, 1).
coefficient and the r-statistic are 12. The file QUARTERLY.XLS contains the U.S. money supply as measured by MI (M1NSA)
y, = 0.467y M + e, and demand deposits reported by commercial banks (DDNSA). Neither series is seasonally
(5.24) adjusted. The series are quarterly averages over the period 1960:01 to 2002:01.
Why is an AR(1) model inadequate? a. Plot the M1NSA sequence against time. Créate the growth rate series and replícate
c. Could an ARMA(1, 1) process gen érate the type of sample ACF and PACF found the results in Section 1 I.
in part a? Estímate the seríes as an ARMA(1,1) process. You should obtain: b. Form the growth rate of the demand deposit series as logCDDNSA^/DDNSA^j). You
Included observations: 99 should find the ACF and PACF are such that
Coefficient Est ím ate Stand ard Error f-Statistic Probability ACF
~ax 0.183 0.159 L15 0.2523 1: 0.0349076 0.0428105 -0.0264426 0.6357616 -0.0792992 -0.0506527
0x _________ 0.510 ________ 0.140 __________ 2L64 _______ 0.0004 7: -0.1681839 0.5227466 -0.1962156 -0.1354325 -0.1646850 0.4345867
Use the Ljung-Box 0-Statistics to show that the ARMA(IJ) model is inadequate. PACF
d. Estímate the series as an AR(2) process to verify the results reported in the text. 1: 0.0349076 0.0416427 -0.0294152 0.6383146 -0.2252285 -0.1047281 7:
10. As you read more of the time-series literature, you will fínd that different authors and -0.1629304 0.2796482 -0.2431228 -0.0673336 0.0813827 0.0557702
different software packages report the AIC and the SBC in various ways. The examples
in the text use AIC = T ln(SSR) + 2/? and SBC = T in(SSR) + n ln(7) where SSR = sum
Explain the observed pattern at lags 4, 8, and 12. What tentative models seem plau-
of squared residuals. However, other common formulas include sible? Does the ACF seem to exhibit geometric decay?
c. Seasonally difference demand deposits using: A4log(DDNSA,) = log(DDNSA), -
AIC = -2 \n{L)!T+ 2n/Tand SBC = -2 \n(L)/T + n ln(T)/r
log(DDNSA),.^. You should find the following:
where L = maximized valué of the likelihood function and
AIC = exp(2/?/7>S57?/r SBC = T n l T -SSR/T ACF
1: 0.9175642 0.7945739 0.6495668 0.4846487 0.3741205 0.2631638
7: 0.1548070 0.0803462 0.0120742 -0.0458305 -0.0836976 -0.1308542
 APPENDIX 2.2: MODEL SELECTION CRITERIA 1 05
104 CHAPTER 2 STATIONARYTIME-SERIES MODELS

máximum likelihood estimation is used. Suppose that {¿t) is a whitc-noise sequence

á: For convenience, leí d, denote the series (I - L)(\ ~ L4)DDNSA,. Find the mosl
drawn trom a normal distribution. The likelihood of any realization s, is
appropriate model for the {dt} series. In particular, compare the following three
models for the series
/ -----T e x P - ^T
d( = a0 + axdt_x + et + P^t-A Model I: AR(l) wiih Seasonal MA
d( = oQ + (I + tfi¿)0 + fl4^4H-i + e
t Model 2: Multiplicative Autoregressive
dt = ÜQ + (i + /?jL)(l + 04LA)£t Model 3: Multiplicative Moving Average Since the sf are tndependent, the likelihood of the joint realizations ¿h <s2, ..., ¿y is

ENDNOTES __________________________________________
n ■ C * P z íL ] 2íT
t=\s¡2ircr j
1. Often, the variance is estimatcd as
If you take the log of the likelihood, you obtain
á2=[l/(r-l)£(yrj)2 T
/=1 l n L = —ln(27r)--ln<72------------- ~Y 1^
2. As discussed in Appendix 2.1, the estimation of lagged MA coeíTicients does not entail a loss of 2 2 2a" ,=l
any usable observations. Henee, the two models are estimated over the same sample period. Now suppose that we observe T valúes of the MA(1) series^ = &st_\ + sr The
3. Most software programs will not be able to estímate (2.43) since there is not a unique set of param-
eter valúes that minimizes the likelihood function. problem is to construct the {et } sequence from the observed valúes of {yt}. If we
4. Some software programs report the Durbin-Watson test statistic as a check for first-order serial cor- knew the true valué of /i and knew that E0 - 0, we could construct eu ..., ET recur-
relation. This well-known test statistic is biased toward fínding no serial correlalion in the presence sively. Given that SQ = 0, it follows that
of lagged dependent variables. Henee, it is usually not used in ARMA models.
5. Estimation of an AR(p) model usually entails a loss of the number of usable observations. Henee, ^l =y\
to estimate a sample using T observations it will be necessary to have (7* + p) observations. Also
note that the procedure outlined necessitates that the second subsample period incorp órate the £-$= y?> - fi£2= yi ~ 0 C^2 - 0y\)
lagged valúes t m, / m_,, ..., í^^. e 4 = y 4 -/3E. 3 =y 4 -0\y 3 -/3(y 2 -/3y l )]
6. Unfortunately, the construction of (2.63) can be sensitive to the choice of q and, if one or more of
the j¡ < 0, the estimated variance can be negative. In such circumstances, it is preferable to use a \n general, et=yt-fkt-\ so that if L is the lag operator
Newey-West (1987) estimator. A number of software packages allow you to directly obtain the t-\
Newey-West estimator of the variance. et = yt /(I +J3L) = 5Z (-/?)'' >V-/
7. To ensure comparability, each was estimated over the 1960:4 - 2002:1 sample period. Note that one /=o
usable observation is lost as a result of diñerencing. In a puré AR(p) model, one observation is also
lost for each lag. Thus, estimation of the AR(2) model can begin no earlier than 1960:4. Note that As long as | (3 \ < 1, the valúes of et will represent a convergent process. This is
no usable observations are lost in estimating MA(g) models. Some software programs initialize the the justification for the assumption that the MA process be invertible. If | ¡3\ > I, we
valúes of fj, ...., eq to be zero so that no additional usable observations are lost. Others will back- cannot represent the {st} series in terms of the observed {y¡} series. If we now sub-
cast the initial valúes of elt ..., eq
stitute the solution for et into the formula for the log likelihood, we obtain
8. In essence, the estimated equation is an ARMA(1, 4) model with the coeíTicients /% and /% con- 2
strained to be equal to zero. In order to distinguish between the two specifications, the notation lnL^zIln(2^)-Ilna ~-^¿(x:(-/?y>V-/
ARMA(1,(1, 4)) or ARMA(1, || 1, 4||) is usedto indícate that only the moving-average terms at lags l ¿ 2 o , =1^=0
1 and 4 are included in the model. Now that we have expressed In L in terms of the observable {yt} series, it is possi-
9. The details of the X-l 1 procedure are not importan! for our purposes. A number of well-known sta-
tistical packages, such as EViews, SAS and RATS, can perform the X-l 1 procedure. The technical ble to select the valúes of /Jand cP- that maximize the valué of In L. Unlike OLS, if you
details of the procedure are explained by the Bureau of the Census (1969). Also note that X-l2 is actually take the partial derivatives of In L with respect to /?and cP- you will not obtain
replacing X-ll. a simple set of first-order conditions. Moreover, the formula becomes much more com-
plicated in higher-order MA(^) processes. Nevertheless, computers can use a number
of available search algorithms to ñnd the valúes of ¿?and cP- that maximize In L.
APPENDIX 2,1: ESTIMATION OF AN MA(1) PROCESS
How do you estimate an MA or an ARMA process? When you estimate a regression
APPENDIX 2.2: -1V1ODEL SELECTION GRITERÍA
using ordinary least squares (OLS), you have a dependent variable and a set of inde- Hypothesis testing is not particularly well suited to testing non-nested models. For
pendent variables. In an AR model, the list of regressors is simply the lagged val úes example, if you wanted to chose between an AR(l) and an MA(2) you could estimate
of the \yt} series. Estimating an MA process is difíerent because you do not know the
valúes of the {£:,} sequence. Since you cannot directly estimate a regression equation,
106 CHAPTER 2 STATIONARYTIME-SERIES MODELS

an ARMA(1, 2) and then try to restriel the MA(2) coefílcients to equal zero. Alternatively, you eould try to
restrict the AR(l) coefficient to equal zero. Nevertheless, the method is unsatisfactory because it necessitates
estimaling the over-parameterized ARMA(l, 2) model. Instead, model selection criteria, such as the AIC and the
SBC, can be used to choose between alternative models. Such model selection criteria can be viewed as
measures of goodness-of-fit that include a cost, .or penalty, for each parameter estimated.
One reason it is not desirable to have an overparameterized model is that forecast error variance increases as a
result of errors arising from parameter estimation. In other words, small models tend to have better out-of-sample
performance than large models. Suppose that the actual data-generating process (DGP) is the AR(l) model

lía is known, the one-step-ahead forecast of j<>/+1 is E¡yi+^ = ayv Henee, the mean squared forecast error is Efyt+\
- ay()2 = Efs*+\ — o2. However, when a is estimated from the data, the one-step-ahead forecast oíyi+i is

where a is the estimated valué of a.

Henee, the mean squared forecast, or prediction, error is
MSPE = £,0',+i - ¿y,)2 = E,[{ay, - ¿y,) + e,+)]2 Since et+\ is independent of a
and yfi it follows that
E t Í3V+1 - oy t ) 2 = E t [(a - o)y t f + a 2

Since Er[(a - á )]2 is strictly positive, parameter uncertainty contributes to forecast error variance in that the
mean squared forecast error exceeds o2. The point is that errors in parameter estimation contribute to forecast
error variance. Moreover, the more parameters estimated, the greater the parameter uncertainty. It is easy to
show that the problem is particularly acute in small samples. Since var^y,) = cP-1 (1 - a2) and, in large samples,
var(¿) = Et[{a — a)]2 P^ (1 - a2)/T \\ follows that
E 1 [(a-a)] 2 {y í ) 2 -¥a 2 ^ [{\ - a 2 ) I T}{\ -a 2 y ]a 2 +a 2

Thus, as T increases, the MSPE approaches a2 The Finite Prediction Error

(FPE) Criteríon
The FPE criterion seeks to minimize the one-step-ahead mean squared prediction error. Now consider the
AR(p) process

If you use the argument in the previous section. the MSPE can be shown to be equal to
 APPENDIX 2.2: MODEL SELECTION CRfTERIA 107

We do not know the true variance o2. However, a2 can be replaced by its unbi-ased estímate SSR/(T — p) to
get
FPE = [l + (p/T))[SSR/(T-p)]
Selectp so as to minimize FPE. We can use logs and note that In(l + p/T) can be approximated by p/T.
Henee, it is possible to select/? to minimize
p/T + ln(SSR) - \n(T-p) which is the same as
minimizing
p+ 71n(SSR)-71n(r-/7)
Since \n(T-p) ~ \nT - p/T, the problem can be written in terms of selectingp so as to minimize
p + rin(SSR) - ln(7) + p which has the same
solution as minimizing
71n(SSR) + 2/7

The AIC and the SBC

The more general AIC selects the (1 + p + q) parameters of an ARMA model so as to maximize the log likelihood
fimction including a penalty for each parameter estimated
AIC = -2 In maximized valué of log likelihood + (1 +p + q)IT
For a given valué of T, selecting the valúes ofp and q so as to minimize in(AIC) is equivalent to selecting p
and q so as to minimize the sum
Tln(SSR) +2(1 +p + q)
Notice that if q - 0 and there is no intercept, this is the result obtained using the FPE. Minimizing the valué
of the AIC implies that each estimated parameter entails a benefit and a cost. Clearly, a beneflt of adding
another parameter is that the valué of SSR is reduced. The cost is that degrees of freedom are reduced and there
is added parameter uncertainty. Thus, adding additional parameters will decrease In (SSR) but will increase (1 +
p + q). The AIC allows you to add parameters until the marginal cost (Le., the marginal cost is 2 for each
parameter estimated) equals the marginal benefit. The SBC incorporates the larger penalty (1 + p + q) \x\T To
use the SBC, select the valúes of p and q so as to minimize
Tin (SSR) + (1 +p + q) ln(7)
For any reasonable sample size, \n(T) > 2 so that the marginal cost of adding parameters using the SBC exceeds
that of the AIC. Henee, the SBC will select a more parsimonious model than the AIC. As indicated in the text, the
SBC has superior large sample properties. It is possible to prove that the SBC is asymptotically consistent while the
AIC is biased toward selecting an overparameterized model. However, Monte Cario