Time Series Analysis and Prediction

Lecture#1
Introduction to Time Series Analysis.

1. Objectives of time series analysis. Examples.

2. Overview of the course.

3. Time series models.

4. Time series modelling: Chasing stationarity.

2
 A Time Series

400

350

300

250

200

150

100

50

0
0 1000 2000 3000 4000 5000 6000 7000

3
 A Time Series

400

350

300

250

200

150

100

50

0
1960 1965 1970 1975 1980 1985 1990
year

4
 A Time Series

SP500: 1960−1990
400

350

300

250

200
$

150

100

50

0
1960 1965 1970 1975 1980 1985 1990
year

5
 A Time Series

SP500: Jan−Jun 1987

340

320

300

280
$

260

240

220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

6
 A Time Series

SP500 Jan−Jun 1987. Histogram

30

25

20

15

10

0
240 250 260 270 280 290 300 310
$

7
 A Time Series

SP500: Jan−Jun 1987. Permuted.

340

320

300

280
$

260

240

220
0 20 40 60 80 100 120

8
 Objectives of Time Series Analysis

1. Compact description of data.

2. Interpretation.

3. Forecasting.

4. Control.

5. Hypothesis testing.

6. Simulation.

9
 Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.

(Makridakis, Wheelwright and Hyndman, 1998)

x 104
12

10

0
0 10 20 30 40 50 60 70 80 90

10
 Transformed data

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

11
 Trend

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

12
 Residuals

1.5

0.5

−0.5

−1
0 10 20 30 40 50 60 70 80 90

13
 Trend and seasonal variation

12

11.5

11

10.5

10

9.5

8.5

7.5

7
0 10 20 30 40 50 60 70 80 90

14
 Objectives of Time Series Analysis

1. Compact description of data.

Example: Classical decomposition: Xt = Tt + St + Yt.
Example: Seasonal adjustment.
2. Interpretation.
Example: Predict sales.
3. Forecasting.

4. Control.

5. Hypothesis testing.

6. Simulation.

15
 Unemployment data

Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994)

x 105
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

16
 Trend

x 105
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

17
 Trend plus seasonal variation

x 105
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

18
 Residuals

x 104
8

−2

−4

−6
1983 1984 1985 1986 1987 1988 1989 1990

19
Predictions based on a (simulated) variable

x 105
8

7.5

6.5

5.5

4.5

4
1983 1984 1985 1986 1987 1988 1989 1990

20
 Objectives of Time Series Analysis

1. Compact description of data:

2. Interpretation. Example: Seasonal adjustment.

Example: Predict unemployment.
3. Forecasting.

4. Control. Example: Impact of monetary policy on unemployment.

5. Hypothesis testing. Example: Global warming.

6. Simulation. Example: Estimate probability of catastrophic events.

21
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

22
 Overview of the Course

1. Time series models

(a) Stationarity.
(b) Autocorrelation function.
(c) Transforming to stationarity.

2. Time domain methods

3. Spectral analysis

4. State space models(?)

23
 Overview of the Course

1. Time series models

2. Time domain methods

(a) AR/MA/ARMA models.
(b) ACF and partial autocorrelation function.
(c) Forecasting
(d) Parameter estimation
(e) ARIMA models/seasonal ARIMA models

3. Spectral analysis

4. State space models(?)

24
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis
(a) Spectral density
(b) Periodogram
(c) Spectral estimation

4. State space models(?)

25
 Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

(a) ARMAX models.
(b) Forecasting, Kalman filter.
(c) Parameter estimation.

26
 Time Series Models

A time series model specifies the joint distribution of the se-

quence {Xt} of random variables.
For example:

P [X1 ≤ x1, . . . , Xt ≤ xt] for all t and x1, . . . , xt.

Notation:
X1, X2, . . . is a stochastic process.
x1, x2, . . . is a single realization.
We’ll mostly restrict our attention to second-order properties only:
EXt, E(Xt1 , Xt2 ).

27
 Time Series Models

2
Example: White noise: Xt ∼ WN (0, σ ).
2
i.e., {Xt} uncorrelated, EXt = 0, VarXt = σ .
Example: i.i.d. noise: {Xt} independent and identically distributed.

P [X1 ≤ x1, . . . , Xt ≤ xt] = P [X1 ≤ x1] · · · P [Xt ≤ xt].

Not interesting for forecasting:

P [Xt ≤ xt|X1, . . . , Xt−1] = P [Xt ≤ xt].

28
 Gaussian white noise

2.5

1.5

0.5

−0.5

−1

−1.5

−2

0 5 10 15 20 25 30 35 40 45 50
−2.5

29
 Gaussian white noise

2.5

1.5

0.5

−0.5

−1

−1.5

−2

0 5 10 15 20 25 30 35 40 45 50
−2.5

30
Moving Averages Filter
 Autoregression

Autoregression is a time series model that uses

observations from previous time steps as input to a
regression equation to predict the value at the next time
step.

successively for t = 1, 2, . . . , 500

 Time Series Models

Example: Binary i.i.d. P [Xt = 1] = P [Xt = −1] = 1/2.

1

0.8

0.6

0.4

0.2

−0.2

−0.4

−0.6

−0.8

−1
0 5 10 15 20 25 30 35 40 45 50

34
 Random walk

Random walk, in probability theory, a process for

determining the probable location of a point subject to
random motions, given the probabilities (the same at
each step) of moving some distance in some direction.

35
 Random walk

Differences: ∇S t = S t− S =t−1
X. t
8

−2

−4
0 5 10 15 20 25 30 35 40 45 50

36
 Random Walk

Recall S&P500 data. (Notice that it’s smooth)

SP500: Jan−Jun 1987
340

320

300

280
$

260

240

220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

37
 Random Walk

Differences: ∇St = St − St−1 = Xt.

SP500, Jan−Jun 1987. first differences
10

2
$

−2

−4

−6

−8

−10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year

38
 Trend and Seasonal Models

5.5

4.5

3.5

2.5
0 50 100 150 200 250

39
 Trend and Seasonal Models

5.5

4.5

3.5

2.5
0 50 100 150 200 250

40
 Trend and Seasonal Models

5.5

4.5

3.5

2.5
0 50 100 150 200 250

41
Trend and Seasonal Models: Residuals

0.5

0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4

0 50 100 150 200 250

−0.5

42
 Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.

2. Transform data so that residuals are stationary.

(a) Estimate and subtract Tt, St.
(b) Differencing.
(c) Nonlinear transformations
(d) Fit model to residuals.

43
 Nonlinear transformations

Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998)

x 104 12
12

11.5

10 11

10.5
8
10

9.5
6

4
8.5

8
2

7.5

0 7
0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90

44
 Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.

2. Transform data so that residuals are stationary.

(a) Estimate and subtract Tt, St.
(b) Differencing.
(c) Nonlinear transformations

3. Fit model to residuals.

45
 Differencing

Recall: S&P 500 data.

SP500: Jan−Jun 1987 SP500, Jan−Jun 1987. first differences
340 10

320
6

4
300

280 0
$

$
−2

260
−4

−6
240

−8

220 −10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year year

46
Task#1

Explore time series datasets.