1

I predict sales for a 1000 products. I use time series cross validation to evaluate forecast for each product.

I want to report how well the model is performing over the whole product range. Thus, the ideal summary would be 1-3 numbers that are easy to interpret.

Product sales vary significantly. Some product are fast sellers with many sales per day. While others are very slow sellers - zero sales for periods of time.

I looked at:

  • MAE, but it suffers from different scales,
  • MAPE, but it's unstable for slow sellers,
  • MASE, but it's hard to interpret and I'm not sure if I can aggregate it.

What metric would be best to capture an overall performance of the model? Or should I split data into fast/medium/slow sellers?

Tautvydas
  • 115
  • 5

1 Answers1

2

I would use a weighted MAE and I would add "missed sale opportunity" if item wasn't present or "unnecessary expenses" if it was there and it wasn't bought for a certain period of time.

Basically, use wasted money as a cost/error function.

Tadas Šubonis
  • 121
  • 3