1

I want to forecast the sales of new products. Last year I used an expert rating to predict the sales of 10 new products. The ratings were used to dicide how many percent of the total sales should be set for every products. So the values add up to 100%. Now I have the real sales data and i want to check if the experts did a better job than just the mean (10% for each of the 10 products). Here is my table: enter image description here

My questions are the following:

  • How can i estimate the goodness of the rating (e.g. mean of the absolute differences, correlation, R2?) also in comparision to the mean values. It should be robust to outliers if possible.

  • is there any way to improve the Planing if i get new ratings for new products this year, for example taking into account systematic errors in the ratings.

here is the replot

enter image description here

here is the errorplot

enter image description here

Thank you so much for any help on this!

Lisa Schmidt
  • 13
  • 2
  • What do you use *percentage* values for? I would expect *absolute* forecasts to be much more useful for subsequent business decisions. For instance, the *percentage* forecasts might be perfect, but total sales turn out to be twice what we expected, so all *absolute* forecasts were half of actuals. – Stephan Kolassa May 14 '20 at 11:21
  • the absolute sales are forecasted elsewhere based of an algorithm I am not aware of. now we want to know - given the total sales estimation - how to distribute production among the products/models – Lisa Schmidt May 14 '20 at 12:27

1 Answers1

0

How can i estimate the goodness of the rating (e.g. mean of the absolute differences, correlation, R2?) also in comparision to the mean values. It should be robust to outliers if possible.

This page gives an overview of commonly used forecast accuracy measures. (I very much recommend the entire book, which is freely available online.)

$R^2$ is not commonly used in assessing forecast accuracy, even though it is equivalent to MSE (which is a common accuracy measure).

One extremely common accuracy measure is the MAPE, which has serious drawbacks. Alternatives are the MSE and the MAE

If you want robustness to outliers, then you will probably like the MAE. Just be aware that this might incentivize biased forecasts, see my answer at the last link above.

is there any way to improve the Planing if i get new ratings for new products this year, for example taking into account systematic errors in the ratings.

You can always record your experts' errors this year and use them to "correct" their forecasts this year... but be aware that this invites gaming. Your expert might now they forecasted too low last year, so they know you are going to increase their forecast this year, but if they think they have already improved their forecasting process and now have an unbiased forecast, they will have an incentive to reduce the forecast before giving it to you so when you increase it, it ends up where the expert wanted it to be in the first place.

Designing a good (and self-improving) forecasting process is difficult, and CV can only be the beginning. You might want to look at Forecast Value Added and/or browse through a few issues of Foresight (full disclosure: I'm an Associate Editor).

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357