Scribd
Upload
10 views4 pages

Forecasting Error Metrics

Uploaded by

umairharoon73
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views4 pages

Forecasting Error Metrics

Uploaded by

umairharoon73
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

COMM 652

Handout #2: Forecasting Error Metrics & Excel

Measuring Forecast Error – Chapter 11 – Page 503


What if we had many periods of forecasts and actual sales to evaluate for accuracy? What should
we do beyond simply eye-balling the accuracy of the forecast method? To begin, write down the
following formulas for measuring forecast error (also known as forecast error metrics). Try and
understand how they are used to quantify forecast error:

1. Mean Absolute Deviation (MAD) =

2. Mean Absolute Percent Error (MAPE) =

3. Mean Square Error (MSE) =

4. Root Mean Square Error (RMSE) =

Exercise 1 – Apply Forecast Accuracy Measures


In this exercise, you will determine which of the 4 forecasting models previously used [i.e., MA=2,
MA=4, SES (0.1) or SES (0.9)] is truly performing the best by calculating MAD, MSE and MAPE for
all 4 forecasting methods. Please complete the table below by hand (calculator and pencil) for
MA = 2.

Month Actual Shed MA=2 Forecast


 error 
Sales (Yt) Forecast( Ŷt ) Error | error | error2   x100
 Yt 
Jan 10 - - - - -
Feb 12 - - - - -
Mar 13 11
Apr 16 12.5
May 19 14.5
June 23 17.5
Totals =

Calculate the “totals” of the 3 right hand side columns, reflect on what they represent and calculate
the following forecast accuracy measures. Note: The denominator of MAD, MSE and MAPE is n
which represents the # of forecast errors. In this example, n = 4.

MAD = MSE = MAPE =


RMSE =

An alternative method to calculate MAD, MSE and MAPE in Excel is to take the average of the 3
right hand side columns. Another valuable Excel functions that can be used here is: =abs( )

© Johnson, M. All rights reserved. 1 of 4


Now do the same for SES (0.9) at home:

Month Actual Shed SES (0.9) Forecast


 error 
Sales (Yt) Forecast ( Ŷt ) Error | error | 2
error   x100
Y
 t 
Jan 10 10
Feb 12 10
Mar 13 11.8
Apr 16 12.88
May 19 15.688
June 23 18.6688
Totals =

MAD = MSE = MAPE =


RMSE =

Please Note: When calculating MAD, MSE and MAPE for SES, we do not include the forecasting
error of the first period (Jan. in this case). The reason for this is because January’s forecast of 10 is
a starting estimate (thus, it would be a forecast error of 0) and therefore we will exclude it when
calculating forecast error. Your Chapter handout also uses the same convention (see Figure 11.8
on page 512)

Please complete the following exercise at home: Now use Excel at Home to create the same table to
calculate the MAD, MSE and MAPE for the Wallace Shed Sales example for MA = 4 and SES (0.1).
When completed, compare the forecasting accuracy measures and draw a conclusion as to which
model is performing the best. Compare your answers to the solutions posted in Moodle in the
filename called “Wallace Gardens posted answers.xlsx”

Important Question: Will your above conclusions with reference to “the best model” always hold
true?

Exercise #2 – Digital Video Recorder (DVR) Sales Forecasting Exercise


This exercise can be carried out in conjunction with your Chapter Handout! It comes from
it! Please try this at home – Mike will provide a brief demo in class.

Part I : Using MA – See Section 11.3 for the same exercise in your textbook!

Download the following file from Moodle: DVR-moving average.xlsx

Create both a 2-month and a 4-month Moving average forecast in columns C and D respectively.

Next, calculate the MSE for both forecasts in cells C28 and D28 to compare these methods. The formula
for calculating MSE in one cell in EXCEL is:

=SUMXMY2(array_x,array_y) / count()
where:
SUMXMY2 stands for SUM of squares of differences of corresponding values in
two arrays. That is, the SUM of X Minus Y to the power of 2!!
array_x is the array of actual sales values
array_y is the array of predicted values

© Johnson, M. All rights reserved. 2 of 4


Compare the 2 MSE values over the last 20 months. (Note: your Chapter handout compares MSE values
over the same periods …. normally we will not do this. Normally we will include all forecast errors in our
calculation of MSE but for the sake of consistency with your Chapter handout, we will proceed doing the
same). Which one has lower error and would be a better forecasting tool? Fill in the following:

MSE:
MA = 2
MA = 4

View the plot of the data and the forecasts using k = 2 and k = 4. Describe in words the difference between
the MA2 forecast line and the MA4 forecast line. Which forecast seems more responsive to the actual
sales?
__________________________________________________________________________
__________________________________________________________________________

Which of the 2 forecasting methods, k = 2 or k = 4, is performing the best only over the last 12 periods (i.e.,
from periods 13 to 24). Hint: Modify your MSE formula so that it only considers the last 12 periods. Which
forecast is performing better? k = 2 or k = 4? Fill in the following:
MSE:
MA = 2
MA = 4
MA = 10
Create a MA forecast using k = 10. Calculate MSE over the last 12 periods and fill in the table above. Draw
a conclusion with respect to the best choice of k for MA applied to the DVR sales.

Exercise #2 Part II – Using SES. See Page 510 for the same exercise!!!

Download the following file from Moodle: DVR-SES.xlsx

Create an exponential smoothing forecast for the DVR Sales data in cells C3 to cells C26. To
begin, let alpha = 0.5. Evaluate the forecasting error using the MSE. See Figure 11.8 on page
512 to guide you if you’re having difficulty.

Which has a higher MSE: alpha = 0.1 or alpha = 0.9?

Which forecast reacts quicker to changes in the sales data: alpha = 0.1 or alpha = 0.9?

Describe in words the difference between the alpha 0.1 forecast line and the alpha 0.9 forecast
line:________________________________________________________.

See Figure 11.9 and 11.10 to determine the optimal value for alpha that minimizes the MSE.
Solver is an “Addin” software that may require you to “add it into Excel”. See pages 5 of this
handout for help “adding it into” Excel.

Once you have entered the solver settings as shown in your Chapter handout as shown in Figure
11.9, press Solve to determine the optimal alpha that minimizes the MSE. Your spreadsheet
should like identical to that of Figure 11.10. The minimized MSE should be 0.268.

Finally, draw a conclusion with respect to the DVR Sales data for:
 the best alpha to be used for Exponential Smoothing

 the best k to use for Moving Average

© Johnson, M. All rights reserved. 3 of 4


DIRECTIONS FOR ADDING IN SOLVER INTO EXCEL:
In Excel, you must “addin” Solver by going to File (on the top toolbar), click onto “Options” at the bottom of
the dialog box as shown:

Next, click onto “Add-Ins” and then “Go” at the bottom of the Window.

Put a check mark in the box to the left of “Solver” and click okay. Go back to the toolbar and click onto
Data. Solver should appear on the top right hand side.

© Johnson, M. All rights reserved. 4 of 4

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy