8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 1/22

Evaluation

The evaluation of Naive Forecasting Techniquesrelies primarily on the comparison of the forecastswith the corresponding actual values

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 2/22

Evaluation Methods

Mean Error (ME)

Mean Absolute Error (MAE)

Mean Squared Error (MSE)

Mean Percentage Error (MPE) Mean Absolute Percentage Error (MAPE)

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 3/22

 

The ME can be very misleading. A ME value of 

zero can mean that the method forecasted the

actual values perfectly (unlikely) or that the

positive and negative errors cancelled eachother out. It tends to Understate the error in

all cases.

n F  A ME 

n

i

t t /

1

  

  

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 4/22

Evaluation Methods

Mean Error (ME) 

Mean Absolute Error (MAE)

Mean Squared Error (MSE) 

Mean Percentage Error (MPE) Mean Absolute Percentage Error (MAPE)

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 5/22

MAE is a way of dealing with the Understatement

of ME. By using the Absolute values of the error,

the mean gives a better indication of the model’s fit.

 MAE A F nt t 

i

n

( ) /1

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 6/22

Evaluation Methods

Mean Error (ME) 

Mean Absolute Error (MAE)

Mean Squared Error (MSE) 

Mean Percentage Error (MPE) Mean Absolute Percentage Error (MAPE)

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 7/22

The MSE eliminates the positive/negativeproblem by squaring the errors. The result

tends to place more emphasis on the larger

errors and, therefore, gives a more conservativemeasure than the MAE.

 MSE A F nt t 

i

n

  

 

2

1

/

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 8/22

  The previous three measures are “seriesspecific;” i.e., they only allow evaluation of 

the series that generated the errors.

The next two measures, by using the

 percentage of the error relative to the actual,are designed to allow comparison of theresults with different models.

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 9/22

Evaluation Methods

Mean Error (ME) 

Mean Absolute Error (MAE)

Mean Squared Error (MSE)

Mean Percentage Error (MPE) Mean Absolute Percentage Error (MAPE)

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 10/22

The MPE is a relative measure of the forecasting

error. It is subject to the “averaging” of the positive

and negative errors.

 MPE A F 

 An

t t 

t t 

n

 

 

 

  

 

1

100 /

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 11/22

Evaluation Methods

Mean Error (ME) 

Mean Absolute Error (MAE)

Mean Squared Error (MSE)

Mean Percentage Error (MPE) 

Mean Absolute Percentage Error (MAPE)

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 12/22

MAPE is a comparative measure that does not have

the problem of averaging the positive and negativeerrors. It is relatively easy to use to communicate a

model’s effectiveness.

 MAPE A F 

 A

nt t 

t t 

n

 

 

 

 

 

 

100

1

/

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 13/22

Measurement of Forecasting Error Mean Error (ME): The average of all the errors of 

forecast for a group of data.

Mean Absolute Error (MAE): The mean, or averageof the absolute values of the errors.

Mean Square Error (MSE): The average of thesquared errors.

Mean Percentage Error (MPE): The average of the percentage errors of a forecast.

Mean Absolute Percentage Error (MAPE): Theaverage of the absolute values of the percentage errorsof a forecast.

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 14/22

Example:

 Nonfarm

Partnership

Tax

Returns:Actual and

Forecast

with = .7

Year Actual Forecast Error

1 1402

2 1458 1402.0 56.0

3 1553 1441.2 111.8

4 1613 1519.5 93.5

5 1676 1584.9 91.1

6 1755 1648.7 106.3

7 1807 1723.1 83.9

8 1824 1781.8 42.2

9 1826 1811.3 14.7

10 1780 1821.6 -41.611 1759 1792.5 -33.5

t t t  F  A F  )1()(1     

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 15/22

Mean Error for the Nonfarm Partnership Forecasted Data

 ME ie

number of forecasts

524 3

10

52 43

.

.

Year Actual Forecast Error

1 1402.0

2 1458.0 1402.0 56.0

3 1553.0 1441.2 111.8

4 1613.0 1519.5 93.55 1676.0 1584.9 91.1

6 1755.0 1648.7 106.3

7 1807.0 1723.1 83.9

8 1824.0 1781.8 42.2

9 1826.0 1811.3 14.7

10 1780.0 1821.6 -41.6

11 1759.0 1792.5 -33.5

524.3

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 16/22

Mean Absolute Error for the Nonfarm Partnership Forecasted Data

 MADie

number of forecasts

6745

10

67 45

.

.

Year Actual Forecast Error |Error|

1 1402.0

2 1458.0 1402.0 56.0 56.0

3 1553.0 1441.2 111.8 111.8

4 1613.0 1519.5 93.5 93.55 1676.0 1584.9 91.1 91.1

6 1755.0 1648.7 106.3 106.3

7 1807.0 1723.1 83.9 83.9

8 1824.0 1781.8 42.2 42.2

9 1826.0 1811.3 14.7 14.7

10 1780.0 1821.6 -41.6 41.6

11 1759.0 1792.5 -33.5 33.5

674.5

 E 

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 17/22

Mean Square Error for the Nonfarm Partnership Forecasted Data

 MSE ie

2

55864 2

10

5586 42

number of forecasts

.

.

Year Actual Forecast Error Error2

1 1402

2 1458 1402.0 56.0 3136.0

3 1553 1441.2 111.8 12499.2

4 1613 1519.5 93.5 8749.75 1676 1584.9 91.1 8292.3

6 1755 1648.7 106.3 11303.6

7 1807 1723.1 83.9 7038.5

8 1824 1781.8 42.2 1778.2

9 1826 1811.3 14.7 214.610 1780 1821.6 -41.6 1731.0

11 1759 1792.5 -33.5 1121.0

55864.2

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 18/22

Mean Percentage Error for the Nonfarm Partnership Forecasted Data

 MPE 

i

i

e

 X  

 

 

 

100

318

10

318%

number of forecasts.

.

Year Actual Forecast Error Error %

1 1402

2 1458 1402.0 56.0 3.8%

3 1553 1441.2 111.8 7.2%

4 1613 1519.5 93.5 5.8%5 1676 1584.9 91.1 5.4%

6 1755 1648.7 106.3 6.1%

7 1807 1723.1 83.9 4.6%

8 1824 1781.8 42.2 2.3%

9 1826 1811.3 14.7 0.8%10 1780 1821.6 -41.6 -2.3%

11 1759 1792.5 -33.5 -1.9%

31.8%

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 19/22

Mean Absolute Percentage Error for the Nonfarm Partnership Forecasted

Data

 MAPE 

i

i

e

 X  

 

 

 

 

100

403

10

4 03%

number of forecasts

.

.

Year Actual Forecast Error |Error %|

1 1402

2 1458 1402.0 56.0 3.8%

3 1553 1441.2 111.8 7.2%

4 1613 1519.5 93.5 5.8%

5 1676 1584.9 91.1 5.4%

6 1755 1648.7 106.3 6.1%

7 1807 1723.1 83.9 4.6%

8 1824 1781.8 42.2 2.3%

9 1826 1811.3 14.7 0.8%

10 1780 1821.6 -41.6 2.3%

11 1759 1792.5 -33.5 1.9%

40.3%

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 20/22

 

Use of Error MeasuresTo identify the best forecasting method

• Use error measure to identify the best value

for the parameters of a specific method.• Use error measure to identify the best

method.

• Use MSE and MAE for both of thesesituations. Note that MSE tends toemphasize large errors.

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 21/22

 

Use of Error Measures, continued  

Forecast bias is the tendency of a

forecasting method to over or under predict.

The mean error, ME, measures the forecast

 bias.

8/23/2019 Forecast Errors

http://slidepdf.com/reader/full/forecast-errors 22/22