Time Series Data

yt = b0 + b1xt1 + . . .+ bkxtk + ut

Dr. Kai Wu
What is a time series?
 Essentially, Time Series is a sequence of numer
ical data obtained at regular time intervals.

 Occurs in many areas: economics, finance, en

vironment, medicine

 The aims of time series analysis are

 to describe and summarize time series data,
 fit models, and make forecasts

2
Importance of time series analysis

 Understand the past

• What happened over the last years, months?
 Forecast the future
• Government wants to know future of unemploy
ment rate, percentage increase in cost of living
etc.
• For companies to predict the demand for the
ir product etc.

3
Why are time series data differe
nt from other data?
 Data are not independent
 Much of the statistical theory relies on the da
ta being independent and identically distribut
ed
 Large sample size is good, but long time series
are not always the best
 Series often change with time, so longer time
series isn’t always better

4
Time Series vs. Cross Sectional
 Time series data has a temporal ordering, un
like cross-section data
 Will need to alter some of our assumptions t
o take into account that we no longer have a
random sample of individuals
 Instead, we have one realization of a stochas
tic (i.e. random) process

5
What Are People Looking for in an Econ
omic Time Series?
 Important features of economic time series in
clude
 Direction
 Turning points
 In addition, we want to see if the series is i
ncreasing/decreasing more slowly/faster th
an it was before

6
Examples of Time series data

 Number of babies born in each hour

 Daily closing price of a stock.
 The monthly trade balance of China for ea
ch year.
 China’s annual GDP

7
Horizons
 Long Term
 5+ years into the future
 R&D, plant location, product planning
 Principally judgement-based

 Medium Term
 1 season to 2 years
 Aggregate planning, capacity planning, sales forecasts
 Mixture of quantitative methods and judgement

 Short Term
 1 day to 1 year, less than 1 season
 Demand forecasting, staffing levels, purchasing, inventory lev
els
 Quantitative methods
8
Example

9
Time Series
 How the data (x) Exports, 1989-1998
and time (t) is re t Year x=Value
corded and prese 1 1989 44,320
2 1990 52,865
nted 3 1991 53,092
4 1992 39,424
5 1993 34,444
6 1994 47,870
7 1995 49,805
8 1996 59,404
9 1997 70,214
10 1998 74,626
10
Time Series

 Coordinates (t,x) Exports

is established in 80,000

the 2 axis 75,000

70,000
 (1, 44,320) 65,000
60,000
 (2, 52,865) 55,000

 (3, 53,092) 50,000

45,000
 etc.. 40,000
35,000
30,000
1988 1990 1992 1994 1996 1998 2000

11
Time Series

 A graphical repre Exports

sentation of time
80,000
series. 75,000
 We use x as a fun 70,000
65,000
ction of t: x= f(t) 60,000
 Data points conn 55,000

ected by a curve 50,000

45,000
40,000
35,000
30,000
1988 1990 1992 1994 1996 1998 2000

12
Time-Series Components

Trend Cyclical

Time-Series

Seasonal Random

13
 Components of Time Series

 Trend (Tt)
 Seasonal variation (St)
 Cyclical variation (Ct)
 Random variation (Rt)

14
 Components of Time Series
Trend (Tt )
 Trend: the long-term patterns or movements
in the data.
 Overall or persistent, long-term upward or d
ownward pattern of movement.
 The trend of a time series is not always linea
r.

15
 Trending Time Series
 Economic time series often have a trend
 Just because 2 series are trending together,
we can’t assume that the relation is causal
 Often, both will be trending because of other
unobserved factors
 Even if those factors are unobserved, we can
control for them by directly controlling for t
he trend

16
Trends (continued)
 One possibility is a linear trend, which can
be modeled as yt = a0 + a1t + et, t = 1, 2, …
 Another possibility is an exponential trend,
which can be modeled as log(yt) = a0 + a1t
+ et, t = 1, 2, …
 Another possibility is a quadratic trend, whi
ch can be modeled as yt = a0 + a1t + a2t2 +
et, t = 1, 2, …
17
Detrending
 Adding a linear trend term to a regression is
the same thing as using “detrended” series i
n a regression
 Detrending a series involves regressing each
variable in the model on t
 The residuals form the detrended series
 Basically, the trend has been partialled out

18
 Detrending (continued)
 An advantage to actually detrending the data
(vs. adding a trend) involves the calculation
of goodness of fit
 Time-series regressions tend to have very hig
h R2, as the trend is well explained
 The R2 from a regression on detrended data b
etter reflects how well the xt’s explain yt

19
 Components of Time Series
Seasonal variation (St )
 Regular periodic fluctuations that occur with
in year.
 Examples:
 Consumption of heating oil, which is high in wi
nter, and low in other seasons of year.
 Gasoline consumption, which is high in summer
when most people go on vacation.

20
 Seasonality
 Often time-series data exhibits some periodic
ity, referred to seasonality
 Example:
 Quarterly data on retail sales will tend to jump u
p in the 4th quarter
 Seasonality can be dealt with by adding a set
of seasonal dummies
 As with trends, the series can be seasonally a
djusted before running the regression
21
 Components of Time Series
Seasonal variation (St )

Summer Summer
30
25
20
15
10
5
0
-5
-10 Winter Winter

22
 Example
Quarterly with Seasonal Components

25

20

15
S a le s

10

0
0 5 10 15 20 25 30 35

Time

23
Seasonal Components Removed
Quarterly without Seasonal Com ponents

25

20

15
S a le s

Y(t)

10

0
0 5 10 15 20 25 30 35
Time

24
Why Do Users Want Seasonally
Adjusted Data?

 Seasonal movements can make features dif

ficult or impossible to see

25
Causes of Seasonal Effects

 Possible causes are

 Natural factors (climate etc.)

 Administrative or legal measures

 Social/cultural/religious traditions (e.g., fix
ed holidays, timing of vacations)

26
 Components of Time Series
Cyclical variation ( Ct )
 Cyclical variations are similar to seasonal
variations. Cycles are often irregular both in
height of peak and duration.
 Examples:
 Long-term product demand cycles.
 Cycles in the monetary and financial sectors.
(Important for economists!)

27
 Cyclical Component
 Long-term wave-like patterns
 Regularly occur but may vary in length
 Often measured peak to peak or trough to trough
1 Cycle
Sales

Year 28
Irregular Component

 Unpredictable, random, “residual” fluctuations

 Due to random variations of
 Nature
 Accidents or unusual events

 “Noise” in the time series

29
Causes of Irregular Effects
 Possible causes
 Unseasonable weather/natural disasters
 Strikes

 Sampling error

30
 Classical Decomposition
 One method of describing a time series
 Decompose the series into various components
 Trend – long term movements in the level of the serie
s
 Seasonal effects – cyclical fluctuations reasonably sta
ble in terms of annual timing (including moving holid
ays and working day effects)
 Cycles – cyclical fluctuations longer than a year
 Irregular – other random or short-term unpredictable f
luctuations

31
Not easy to understand the pattern!

32
Our aim
 Understand and identify different variations
so that we can easily predict the future varia
tions separately and combine together

 Look how the above complicated series cou

ld be understood as follows separately

33
34
35
36
Can you imagine how all components aggregate
together to form this?

37
Multiplicative Time-Series Model
for Annual Data
 Used primarily for forecasting
 Observed value in time series is the product
of components
Yi  Ti  Ci  Ii
where Ti = Trend value at year i
Ci = Cyclical value at year i
Ii = Irregular (random) value at year i
38
Multiplicative Time-Series Model
with a Seasonal Component
 Used primarily for forecasting
 Allows consideration of seasonal variation

Yi  Ti  Si  Ci  Ii
where Ti = Trend value at time i
Si = Seasonal value at time i
Ci = Cyclical value at time i
Ii = Irregular (random) value at time i
39
 Time Series Data

Describing the dynamics

of a Time Series

40
Common Processes

 Moving Average (MA) processes

 Autoregressive (AR) processes
 Autoregressive / Moving Average (ARMA) pr
ocesses
 Autoregressive conditional heteroscedastic (A
RCH) processes

41
Moving Average Process
Moving Average Method
 Series of arithmetic means
 Used only for smoothing
 Provides overall impression of data over time

43
 Smoothing Techniques
 Smoothing helps to see overall patterns in ti
me series data.
 Smoothing techniques smooth or “iron” out
variation to get the overall picture.
 There are several smoothing techniques of t
ime series.

44
Smoothing Techniques:
Moving Average (MA)
 Odd number of points. Points (k) – length for c
omputing MA
 k=3
y1  y2  y3
MA1 
3
y 2  y3  y 4
MA2 
3
and so on.

45
Smoothing Techniques:
Moving Average (MA) k=3
Year Series 3 Point MA
1990 5
1991 6 6 MA1
1992 7
1993 8 MA2
1994 10
1995 11
1996 12
1997 12
1998 12 12.0
1999 12 12.3
2000 13 12.7
2001 13
46
Smoothing Techniques:
Moving Average (MA) k=3
Moving Average (3)

Actual Forecast
14
13
12
11
10
9
8
7
6
5
4
3
1 2 3 4 5 6 7 8 9 10 11 12

47
Smoothing Techniques:
Moving Average (MA) k=5
Year Series 5 Point MA
1990 5
1991 6
1992 7 7.2
1993 8
1994 10
1995 11
1996 12
1997 12
1998 12
1999 12 12.4
2000 13
2001 13

48
Smoothing Techniques:
Moving Average (MA) k=5
Moving Average (5)
Actual Forecast
14
12
10
8
6
4
2
0
1 2 3 4 5 6 7 8 9 10 11

49
Moving Average Graph
Sales
8 Actual
6
4
2
0
93 94 95 96 97 98
Year
50
Moving Average [An Example]

 You work for Firestone Tire. You want to

smooth random fluctuations using a 3-pe
riod moving average.

199520,000
1996 24,000
199722,000
199826,000
199925,000
51
Moving Average [Solution]

Year Sales MA(3) in 1,000

1995 20,000 NA
1996 24,000 (20+24+22)/3 = 22
1997 22,000 (24+22+26)/3 = 24
1998 26,000 (22+26+25)/3 = 24
1999 25,000 NA

52
 Moving Average
Year Response Moving
Ave
Sales
1994 2 NA
1995 5 3 8
1996 2 3 6
1997 2 3.67 4
1998 7 5 2
1999 6 NA 0

94 95 96 97 98 99

53
Moving Average Processes
 MA(1): First Order MA process
Yt     t   t 1
 “Moving Average”
 Yt is constructed from a weighted sum of the t
wo most recent values of  .

54
Higher Order MA Process

 MA(q): qth order moving average process

Yt     t  1 t 1   2 t  2     q t  q

55
Autoregressive Process
 Autoregressive Model
 Used for forecasting trend
 Like regression model
 Independent variables are lagged response varia
bles Yi-1, Yi-2, Yi-3 etc.
 Assumes data are correlated with past data
values
 1st Order: Correlated with prior period
 Estimate with ordinary least squares
57
Autoregressive Model [An Example]
 The Office Concept Corp. has acquired a number
of office units (in thousands of square feet) over
the last 8 years. Develop the 2nd order
Autoregressive models.
Year Units
92 4
93 3
94 2
95 3
96 2
97 2
98 4
99 6
58
 Autoregressive Model [Example Solu
tion]
•Develop the 2nd order table Year Yi Yi-1 Yi-2
•Use Excel to run a regression 92 4 --- ---
model 93 3 4 ---
94 2 3 4
95 3 2 3
Excel Output 96 2 3 2
Coefficients 97 2 2 3
98 4 2 2
Intercept 3.5
99 6 4 2
X Variable 1 0.8125
X Variable 2 -0.9375
Yi  3.5  .8125Yi  1  .9375Yi  2
59
AR Process
 The (unconditional mean) for an AR(1) pr
ocess, with a constant (μ) is given by:

yt    1 yt 1  ut  AR(1)

E ( yt ) 
1  1

60
Higher Order AR Processes
 pth order autoregression: AR(p)

Yt  c  1Yt 1  2Yt  2     pYt  p   t

 Stationarity: We will assume that the roots of th
e following all lie outside the unit circle.

1  1 z  2 z     p z  0
2 p

61
How to determining the order of
MA and AR process?

62
Autocorrelation Function
 How correlated is the series with itself at vario
us lag values?
 E.g. If you plot Yt+1 versus Yt and find the corr
elation, that's the correl. at lag 1
 ACF lets you calculate all these correls. witho
ut plotting at each lag value.
 ACF is a basic building block of time series an
alysis.
63
 ACF of an MA process
 Drops to zero after la
ACF

g=q 0 .8

0 .6

 That's a good way to 0 .4

0 .2

determine what q sh 0
0 5 10 15 20 25 30

ould be!
-0 .2

-0 .4

-0 .6

-0 .8

-1

64
 ACF of an AR process
 Never completely dies of
f, not useful for finding or ACF

der p. 1

0 .8

 AR(1) has exponential de 0 .6

cay in ACF
0 .4

0 .2

 Instead, use Partial ACF 0

0 5 10 15 20 25 30
-0 .2
(PACF), which dies after l -0 .4

ag=p -0 .6

-0 .8
 PACF of MA never dies. -1

65
 ACF PACF
MA Drop to zero Never drops
after lag q to zero
AR Never drops Drop to zero
to zero after lag p

66
Mixed Autoregressive Moving Av
erage Processes (ARMA)
 ARMA(p,q) includes both autoregressive a
nd moving average terms

Yt  c  1Yt 1  2Yt  2     pYt  p   t

 1 t 1   2 t  2     q t  q

67
ARMA

 ARMA(p,q) combines AR and MA

 Often p,q <= 1 or 2

68
ARIMA
 AR-Integrated-MA
 ARIMA(p,d,q)
 d=order of differencing before applying ARM
A(p,q)
 For nonstationary data with stochastic trend

69
Identification
 Identification of the most appropriate model
is the most important part of the process, w
here it becomes as much ‘art’ as ‘science’.

 The first step is to determine if the variable

is stationary, this can be done with the ACF.
If it is not stationary it needs to be first-diffe
renced. (it may need to be differenced again
to induce stationarity)

70
Identification
 The next stage is to determine the p and q
in the ARIMA (p, I, q) model (the I refers
to how many times the data needs to be di
fferenced to produce a stationary series)

71
Identification
 To determine the appropriate lag structure in the A
R part of the model, the PACF is used, where the n
umber of non-zero points of the PACF determine w
here the AR lags need to be included.
 To determine the MA lag structure, the ACF is use
d, again the non-zero points suggest where the lags
should be included.
 Seasonal dummy variables may also need to be inc
luded if the data exhibits seasonal effects.

72
 Forecasting
 One of the most important tests of how well a model
performs is how well it forecasts.
 One of the most useful models for forecasting is the
ARIMA model.
 To produce dynamic forecasts the model needs to in
clude lags of either the variables or error terms.

73
Forecasting

74
 Types of Forecast
 Forecasts can be either in-sample or out-of-s
ample forecasts.
 In general the out-of-sample forecasts are a
better test of how well the model works, as t
he forecast uses data not included in the esti
mation of the model.
 To conduct out-of-sample forecasts, we nee
d to leave some observations at the end of o
ur sample for this purpose
75
 Types of Forecasts
 A one-step-ahead is a forecast for the next observatio
n only.
 A multi-step-ahead forecast is for 1,2,3,…s steps ahe
ad.
 A recursive window for the forecast means that the i
nitial estimation date is fixed but the additional obser
vations are added one by one to the estimation time s
pan.
 A rolling window is where the estimation time perio
d is fixed but the start and end dates successively inc
rease by 1.

76
Conditional Forecasting
 A conditional expectation is one that is taken for ti
me t + 1, conditional upon or given all information
available up to and including time t (this is import
ant later). It is written as:

E ( yt 1 | t )
t  all informatio n at time t

77
 Measuring Forecast Accuracy
 To determine how accurate a forecast is, the simples
t method is to plot the forecast against the actual val
ues as a direct comparison
 In addition it may be worthwhile to compare the tur
ning points, this is particularly important in finance.
 There are a number of methods to determine accura
cy of the forecast, often more than one is included i
n a set of results.

78
 Tests of Forecast Accuracy
 Tests of forecast accuracy are based on the differe
nce between the forecast of the variables value at
time t and the actual value at time t. The closer th
e two are together and the smaller the forecast err
or, the better the forecast.
 There are a variety of statistics measuring this acc
uracy, mostly based on an average of the errors be
tween the actual and forecast values at time t.

79
Mean Squared Error (MSE)
 The MSE statistic can be defined as:

1 T
MSE   t  s t ,s
T  (T1  1) t T1
( y  f ) 2

T  total sample size

T1  first out - of - sample forecast observatio n
f t , s  s - step - ahead forecasts made at time t
yt  s  the actual value at time t

80
MSE Example
Steps Forecast Actual Squared
Ahead Error
1 0.1 0.15 0.0025
2 0.25 0.20 0.0025
3 0.5 0.40 0.01

MSE  (0.0025  0.0025  0.01) / 3  0.005

81