Time Series Forecasting

Applications

• Retail stores: forecast sales, …

• Energy companies: forecast reserves, production, demand, prices, …
• Transportation companies: forecast future travels, …
• Banks: forecast new home purchases, …
• Etc.
 Introduction

• Main purpose of forecasting:

Reduce uncertainty and make better estimates of what will happen in
the future based on historical data

• Subjective methods
• Methods based on intuition, experience

• More formal quantitative techniques (moving average, exponential

smoothing, etc.)
 Descriptive vs. Predictive Modelling

Time Series Analysis Time Series Forecasting

• descriptive modelling • predictive modelling

• time series is modeled to determine its • uses the information in a time series to

components in terms of seasonal forecast future values of that series

patterns, trends, etc.

Time Series: Definition and Creation
 Definition

• Definition: An ordered sequence of values of a variable at equally spaced time intervals

indexed set of
equally spaced
measures

2007 2019
 Accumulation and Interpretation

Missing,
Accumulation Zero Value
Interpretation

time series interpreted

time-stamped data
time series
 Accumulation – Interval Selection

Date Website Visits

accumulate
03/09/2022 11 monthly interval

08/09/2022 14
12/09/2022 7
19/09/2022 17
Date Website Visits
23/09/2022 5
09/2022 -
01/10/2022 12
10/2022 -
09/10/2022 8
14/10/2022 13

transactional 20/10/2022 15
data
 Accumulation – Method within Interval

Date Website Visits

03/09/2022 11 Sum method

08/09/2022 14
12/09/2022 7
19/09/2022 17
Date Website Visits
23/09/2022 5
09/2022 54
01/10/2022 12
10/2022 48
09/10/2022 8
14/10/2022 13

transactional 20/10/2022 15
data
 Accumulation – Method within Interval

Date Website Visits

03/09/2022 11 average method

08/09/2022 14
12/09/2022 7
19/09/2022 17
Date Website Visits
23/09/2022 5
09/2022 10.8
01/10/2022 12
10/2022 12
09/10/2022 8
14/10/2022 13

transactional 20/10/2022 15
data
 Accumulation – Method within Interval

Date Website Visits Date Website Visits

01/2022 24 01/2022 24

02/2022 11 02/2022 11

03/2022 54 03/2022 54

04/2022 0 04/2022 40

05/2022 64 05/2022 64

06/2022 . 06/2022 40

07/2022 46 07/2022 46

08/2022 27 08/2022 27

09/2022 54 09/2022 54

Accumulated Interpreted
time series time series
Time Series Components
 Time Series Components

Time series components:

• Level: describes the average value of the series

• Trend: long-term movement or direction of the data over time

• Seasonality: repeating patterns that occur at regular intervals within the data

• Cyclic variations: longer-term patterns that are not as regular as seasonality

• Noise : random fluctuations due to various factors, including measurement error

Time Series Components
 Trend

• A trend exists when there is a long-term increase or decrease in the data

• Trends can be linear (constant slope) or non-linear (curved or exponential)
 Seasonality

• Repeating patterns that occur at regular intervals within the data.

• Seasonality can be observed in daily, weekly, monthly, quarterly, or annual patterns. It is typically
associated with calendar-related effects
When handling long time series
 Cyclic Variations

• A cycle occurs when the data exhibit rises and falls that are not of a fixed frequency.
• These fluctuations are often associated with economic, political, or societal factors and may
span multiple seasons or years
 Seasonal or Cyclic?

Seasonal Cyclic

shorter-term patterns: longer-term fluctuations: multiple

Duration
repeat at fixed and regular intervals seasons or years

Pattern consistent and predictable pattern variable pattern

regularity, often on a daily, weekly, irregular and do not occur at fixed

Regularity
monthly, or quarterly basis. or predictable intervals

calendar-related effects such as weather economic, political, or societal

Causes
changes, holidays factors

Magnitude smaller magnitude fluctuations larger magnitude fluctuations

 Examples

• strong seasonality within each year

• strong increasing trend
• some strong cyclic behaviour with a
• strong seasonality
period of about 6–10 years
• no evidence of any cyclic behaviour
• no apparent trend over this period
Time Series Patterns - Examples

• no seasonality
• downward trend
• long cycle?

• No trend, seasonality or cyclic behaviour

• random fluctuations
Time Series Forecasting Steps
 Time Series Forecasting Steps

1. Data collection
 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing
 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection
 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection

6. Model training
 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation
 Time Series Forecasting Steps

1. Data collection

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation

8. Model tuning
 Time Series Forecasting Steps

1. Data collection 9. Forecasting

2. Exploratory Data Analysis (EDA)

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation

8. Model tuning
 Time Series Forecasting Steps

1. Data collection 9. Forecasting

2. Exploratory Data Analysis (EDA) 10. Model Validation

3. Data preprocessing

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation

8. Model tuning
 Time Series Forecasting Steps

1. Data collection 9. Forecasting

2. Exploratory Data Analysis (EDA) 10. Model Validation

3. Data preprocessing 11. Model Deployment

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation

8. Model tuning
 Time Series Forecasting Steps

1. Data collection 9. Forecasting

2. Exploratory Data Analysis (EDA) 10. Model Validation

3. Data preprocessing 11. Model Deployment

4. Data partitioning: Train-test split

5. Model selection

6. Model training

7. Model evaluation

8. Model tuning
Model Evaluation
 Model Evaluation

• Common metrics include MAE, MSE, RMSE, MAPE

• Visualization: examining time plots of the actual and predicted series can
shed light on performance and hint toward possible improvements
 Naïve Forecasts

• A naive forecast is the most recent value of the series: at time t, the forecast
for any future period t+k is simply the value of the series at time t

• Compare the selected forecasting method against the naive-forecasting

approach
 Model Evaluation: MAE

Compare forecasted values with actual values

• Forecast error = Actual value – Forecast value

Þ Measure of accuracy: Mean absolute error (MAE):

MAE
 Example: MAE
ACTUAL ABSOLUTE VALUE OF
YEAR SALES OF WIRELESS FORECAST SALES ERRORS (DEVIATION),
SPEAKERS (ACTUAL – FORECAST)

1 110 —
2 100 110
3 120 100
4 140 120
5 170 140
6 150 170
7 160 150
8 190 160
9 200 190
10 190 200
11 — 190
 Example: MAE
ACTUAL FORECAST SALES ABSOLUTE VALUE OF
YEAR SALES OF WIRELESS ERRORS (DEVIATION),
SPEAKERS naïve model
(ACTUAL – FORECAST)

1 110 —
2 100 110
3 120 100
4 140 120
5 170 140
6 150 170
7 160 150
8 190 160
9 200 190
10 190 200
11 — 190
 Example: MAE
ACTUAL FORECAST SALES ABSOLUTE VALUE OF
YEAR SALES OF WIRELESS ERRORS (DEVIATION),
SPEAKERS naïve model
(ACTUAL – FORECAST)

1 110 — —
2 100 110 |100 – 110| = 10
3 120 100 |120 – 100| = 20
4 140 120 |140 – 120| = 20
5 170 140 |170 – 140| = 30
6 150 170 |150 – 170| = 20
7 160 150 |160 – 150| = 10
8 190 160 |190 – 160| = 30
9 200 190 |200 – 190| = 10
10 190 200 |190 – 200| = 10
11 — 190 —
Sum of |errors| = 160
MAE = 160/9 = 17.8
 Example: MAE
ACTUAL ABSOLUTE VALUE OF
YEAR SALES OF WIRELESS FORECAST SALES ERRORS (DEVIATION),
SPEAKERS (ACTUAL – FORECAST)

1 110 — —
2 100 110 |100 – 110| = 10
3 120 100 |120 – 100| = 20
4 140 120 |140 – 120| = 20
5 170 140 |170 – 140| = 30
∑ 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 𝑒𝑟𝑟𝑜𝑟 160
6 150 𝑀𝐴𝐸 = 170 = = 17.8
|150 – 170| = 20
𝑛 9
7 160 150 |160 – 150| = 10
8 190 160 |190 – 160| = 30
9 200 190 |200 – 190| = 10
10 190 200 |190 – 200| = 10
11 — 190 —
Sum of |errors| = 160
MAE = 160/9 = 17.8
 Measures of Forecast Accuracy

MSE ∑(𝑒𝑟𝑟𝑜𝑟)!
𝑀𝑆𝐸 =
𝑛

MAPE
Accuracy as a percentage of the error
𝑒𝑟𝑟𝑜𝑟
∑
𝑀𝐴𝑃𝐸 = 𝑎𝑐𝑡𝑢𝑎𝑙 x 100%
𝑛

• BIAS
Tells if the forecast it too high/low and by how much ∑ 𝑒𝑟𝑟𝑜𝑟
𝐵𝐼𝐴𝑆 =
𝑛
Model Selection
 Model Selection

Forecasting Methods

Smoothing Regression Machine Learning

Methods Methods Models

Simple Simple Random

Moving Avg Regression Forests

Weighted Multiple Neural

Moving Avg Regression Networks

Exponential
ARIMA …
Smoothing

… …
Model Selection: Smoothing Methods
Smoothing Methods
Moving Average
Moving Average
Centered Moving Average (CMA)

n=5

t-2 t-1 t t+1 t+2

 Centered Moving Average (CMA)

MONTH ACTUAL SHED SALES 3-MONTH CMA

January 10
February 12 (10 + 12 + 13)/3 = 11.67
March 13 (12 + 13 + 16)/3 = 13.67
April 16 (13 + 16 + 19)/3 = 16.00
May 19 (16 + 19 + 23)/3 = 19.33
June 23 (19 + 23 + 26)/3 = 22.67
July 26 (23 + 26 + 30)/3 = 26.33
August 30 (26 + 30 + 28)/3 = 28.00
September 28 (30 + 28 + 18)/3 = 25.33
October 18 (28 + 18 + 16)/3 = 20.67
November 16 ??
December 14
 Simple Moving Average (SMA)

The next forecast is the average of the most recent n data values from the time
series

Sum of demands in previous n periods

Moving average forecast =
n
Simple Moving Average (SMA)

n values

𝑌" + 𝑌"%$ + … + 𝑌"%&#$

𝐹"#$ =
𝑛
 Example: Wallace Garden Supply

Wallace Garden Supply wants to forecast the monthly demand for its Storage
Shed

• Collected data for the past year

• Use a three-month moving average (n = 3)

 Example: Wallace Garden Supply

MONTH ACTUAL SHED SALES 3-MONTH MOVING AVERAGE

January 10
February 12
March 13
April 16
May 19
June 23
July 26
August 30
September 28
October 18
November 16
December 14
January —
 Example: Wallace Garden Supply

MONTH ACTUAL SHED SALES 3-MONTH MOVING AVERAGE

January 10
February 12
March 13
April 16 (10 + 12 + 13)/3 = 11.67
May 19 (12 + 13 + 16)/3 = 13.67
June 23 (13 + 16 + 19)/3 = 16.00
July 26 (16 + 19 + 23)/3 = 19.33
August 30 (19 + 23 + 26)/3 = 22.67
September 28 (23 + 26 + 30)/3 = 26.33
October 18 (26 + 30 + 28)/3 = 28.00
November 16 (30 + 28 + 18)/3 = 25.33
December 14 (28 + 18 + 16)/3 = 20.67
January — ???
 Example: Wallace Garden Supply

MONTH ACTUAL SHED SALES 3-MONTH MOVING AVERAGE

January 10
February 12
March 13
April 16 (10 + 12 + 13)/3 = 11.67
May 19 (12 + 13 + 16)/3 = 13.67
June 23 (13 + 16 + 19)/3 = 16.00
July 26 (16 + 19 + 23)/3 = 19.33
August 30 (19 + 23 + 26)/3 = 22.67
September 28 (23 + 26 + 30)/3 = 26.33
October 18 (26 + 30 + 28)/3 = 28.00
November 16 (30 + 28 + 18)/3 = 25.33
December 14 (28 + 18 + 16)/3 = 20.67
January — (18 + 16 + 14)/3 = 16.00
 Weighted Moving Average (WMA)

Weighted moving averages use weights to put more emphasis on previous

periods

• Often used when a trend is emerging

∑ (Weight in period i)(Actual value in period)

Ft+1 =
∑(Weights)

• Mathematically: w1Yt + w2Yt−1 +... + wnYt−n+1

Ft+1 =
w1 + w2 +... + wn
If wi = 1 … ?
where wi = weight for the ith observation
 Example: Wallace Garden Supply

WEIGHTS APPLIED PERIOD

3 Last month

2 Two months ago

1 Three months ago

3 x Sales last month + 2 x Sales two months ago + 1 x Sales three months ago

Sum of the weights

 Example: Wallace Garden Supply

MONTH ACTUAL SHED SALES 3-MONTH MOVING AVERAGE

January 10
February 12
March 13
April 16 [(3 X 13) + (2 X 12) + (10)]/6 = 12.17

May 19 [(3 X 16) + (2 X 13) + (12)]/6 = 14.33

June 23 [(3 X 19) + (2 X 16) + (13)]/6 = 17.00

July 26 [(3 X 23) + (2 X 19) + (16)]/6 = 20.50

August 30 [(3 X 26) + (2 X 23) + (19)]/6 = 23.83

September 28 [(3 X 30) + (2 X 26) + (23)]/6 = 27.50

October 18 [(3 X 28) + (2 X 30) + (26)]/6 = 28.33

November 16 [(3 X 18) + (2 X 28) + (30)]/6 = 23.33

December 14 [(3 X 16) + (2 X 18) + (28)]/6 = 18.67

January — ???
 Example: Wallace Garden Supply

MONTH ACTUAL SHED SALES 3-MONTH MOVING AVERAGE

January 10
February 12
March 13
April 16 [(3 X 13) + (2 X 12) + (10)]/6 = 12.17

May 19 [(3 X 16) + (2 X 13) + (12)]/6 = 14.33

June 23 [(3 X 19) + (2 X 16) + (13)]/6 = 17.00

July 26 [(3 X 23) + (2 X 19) + (16)]/6 = 20.50

August 30 [(3 X 26) + (2 X 23) + (19)]/6 = 23.83

September 28 [(3 X 30) + (2 X 26) + (23)]/6 = 27.50

October 18 [(3 X 28) + (2 X 30) + (26)]/6 = 28.33

November 16 [(3 X 18) + (2 X 28) + (30)]/6 = 23.33

December 14 [(3 X 16) + (2 X 18) + (28)]/6 = 18.67

January — [(3 X 14) + (2 X 16) + (18)]/6 = 15.33

Choosing the Window Width (CMA)
 Choosing the Window Width (SMA)

• domain knowledge in terms of relevance of past values and how fast the
series changes

• empirical predictive evaluation via experimenting with different values of n

and comparing MAE
Simple Exponential Smoothing
 Simple Exponential Smoothing

• A type of moving average

• Easy to use

• Requires little record keeping of data

New forecast = Last period’s forecast

+ a (Last period’s actual demand
– Last period’s forecast)

where a is a weight (or smoothing constant) with a value 0 ≤ a ≤ 1

 Simple Exponential Smoothing

Ft+1 = Ft + α (Yt − Ft )

where
Ft+1 = new forecast (for time period t + 1)
Ft = previous forecast (for time period t)
a = smoothing constant (0 ≤ a ≤ 1)
Yt = previous period’s actual demand

The idea is simple:

new estimate = old estimate + some fraction of the error in the last period
Simple Exponential Smoothing

Ft+1 = Ft + α (Yt − Ft )

𝐹"#$ = 𝐹" + 𝛼(𝑌" - 𝐹" )

= 𝛼𝑌" + (1 − 𝛼)𝐹"
= 𝛼𝑌" + (1 − 𝛼) 𝐹"%$ + 𝛼(𝑌"%$ − 𝐹"%$ )
= 𝛼𝑌" + 1 − 𝛼 𝛼𝑌"%$ + (1 − 𝛼)! 𝐹"%$
= …
 Example

• In January, February’s demand for a car model was predicted to be 142

• Actual February demand was 153 cars

Þ Using a smoothing constant of α = 0.20, what is the forecast for March?

New forecast (for March demand) = 142 + 0.2 x (153 – 142)
= 144.2 or 144 cars

Þ If actual March demand = 136:

New forecast (for April demand) = 144.2 + 0.2 x (136 – 144.2)
= 142.6 or 143 cars
 Selecting the Smoothing Constant α

• set by the user, determines the rate of learning

• value close to 1 indicates fast learning (that is, only the most recent values
have influence on forecasts)

• value close to 0 indicates slow learning (past values have a large influence
on forecasts)

• choice depends on required amount of smoothing, and on how relevant

the history is for generating forecasts
 Selecting the Smoothing Constant

The general approach is to develop trial forecasts with different values of α

and select the α that results in the lowest MAE
 Example - Port of Baltimore

Exponential Smoothing Forecast for a = 0.1 and a = 0.5

FORECAST
ACTUAL TONNAGE FORECAST
QUARTER USING
UNLOADED USING a = 0.10
a = 0.50

1 180 175 175

2 168 175.5 = 175.00 + 0.10(180 – 175) 177.5

3 159 174.75 = 175.50 + 0.10(168 – 175.50) 172.75

4 175 173.18 = 174.75 + 0.10(159 – 174.75) 165.88

5 190 173.36 = 173.18 + 0.10(175 – 173.18) 170.44

6 205 175.02 = 173.36 + 0.10(190 – 173.36) 180.22

7 180 178.02 = 175.02 + 0.10(205 – 175.02) 192.61

8 182 178.22 = 178.02 + 0.10(180 – 178.02) 186.30

9 ? 178.60 = 178.22 + 0.10(182 – 178.22) 184.15

 Example - Port of Baltimore
Absolute Deviations and MAEs
ACTUAL FORECAST ABSOLUTE FORECAST ABSOLUTE
QUARTER TONNAGE WITH DEVIATIONS FOR WITH DEVIATIONS FOR
UNLOADED a = 0.10 a = 0.10 a = 0.50 a = 0.50
1 180 175 5 175 5

2 168 175.5 7.5 177.5 9.5

3 159 174.75 15.75 172.75 13.75

4 175 173.18 1.82 165.88 9.12

5 190 173.36 16.64 170.44 19.56

6 205 175.02 29.98 180.22 24.78

7 180 178.02 1.98 192.61 12.61

8 182 178.22 3.78 186.30 4.3

Sum of absolute deviations 82.45 98.63

Σ|errors|
MAE = !
= 10.31 MAE = 12.33
 Example - Port of Baltimore
Absolute Deviations and MAEs
ACTUAL FORECAST ABSOLUTE FORECAST ABSOLUTE
QUARTER TONNAGE WITH DEVIATIONS FOR WITH DEVIATIONS FOR
UNLOADED a = 0.10 a = 0.10 a = 0.50 a = 0.50
1 180 175 5 175 5

2 168 175.5 7.5 177.5 9.5

3 159 174.75 15.75 172.75 13.75

4 175 173.18 1.82 165.88 9.12

5 190 173.36 16.64 170.44 19.56

6 205 175.02 29.98 180.22 24.78

7 180 178.02 1.98 192.61 12.61

8 182 178.22 3.78 186.30 4.3

Sum of absolute deviations 82.45 98.63

Σ|errors|
MAE = !
= 10.31 MAE = 12.33
Advanced Exponential Smoothing
Model Selection: Regression Methods
 Model with Seasonality

• For seasonality: create a new categorical variable that denotes the season for each
value

• This categorical variable is then

turned into dummies, which in turn
are included as predictors in the
regression model
Model with Trend and Seasonality

• create a model that captures both trend

and seasonality by including predictors of
both types

• 13 predictors:
• 11 dummies for month,
• and x and x2 for trend
ARIMA
ARIMA
 Stationarity of a time series models

The process of transformation (non-stationary to

stationary) is referred to as integration

https://www.oreilly.com/library/view/hands-on-machine-learning/9781788992282/15c9cc40-
bea2-4b75-902f-2e9739fec4ae.xhtml