7.4 Forecast evaluation
We can compute the forecast performance metrics as usual.
fit.ets <- forecast::ets(traindat, model="AAA")
fr <- forecast::forecast(fit.ets, h=12)
Look at the forecast so you know what years and months to include in your test data. Pull those 12 months out of your data using the window()
function.
testdat <- window(chinookts, c(1996,1), c(1996,12))
Use accuracy()
to get the forecast error metrics.
forecast::accuracy(fr, testdat)
## ME RMSE MAE MPE MAPE
## Training set -0.0001825075 0.5642326 0.4440532 -9.254074 25.40106
## Test set 0.3143200919 0.7518660 0.6077172 65.753096 81.38568
## MASE ACF1 Theil's U
## Training set 0.7364593 0.07490341 NA
## Test set 1.0078949 0.05504107 0.4178409
We can do the same for the ARIMA model.
fit <- forecast::auto.arima(traindat)
fr <- forecast::forecast(fit, h=12)
forecast::accuracy(fr, testdat)
## ME RMSE MAE MPE MAPE MASE
## Training set 0.01076412 0.5643352 0.3966735 -1.219729 26.91589 0.6578803
## Test set 0.79665978 0.9180939 0.7966598 19.587692 53.48599 1.3212549
## ACF1 Theil's U
## Training set -0.05991122 NA
## Test set -0.12306276 0.5993699