1

I made some predictions with my model and measured its performance using the RMSE. I cannot compare the errors between predictions because some time series have different scales. Some in the 1000's other in the 1000000's.

So I decided to use the RMSPE as stated in RMSE is scale-dependent; is RMSE%?

My problem is that some of my time series data contain zeros in it as legitimate values and the RMSPE occasionally gives a division by zero.

I thought of adding a very small value like 0.000001 to the denominator just to prevent the division by zero but is there a better way of doing this?

Marcus
  • 255
  • 1
  • 4

1 Answers1

3

Dividing by zero, or even small positive values, will cause problems. Instead, I suggest you use scaled errors as discussed here: https://otexts.com/fpp3/accuracy.html#scaled-errors

The RMSSE (root mean scaled squared error) should do what you want.

Rob Hyndman
  • 51,928
  • 23
  • 126
  • 178
  • I implemented the RMSSE and still got a division by zero. Some of my data is zero and I've divided it into chunks for cross-validation meaning that some of the chunks only contain 0s. The average distance between 0s is still 0. – Marcus Feb 12 '21 at 09:34
  • 1
    If you have training data that consist entirely of zeros, then you can't scale any error measure. I suggest you just omit those series. – Rob Hyndman Feb 13 '21 at 00:45