2

I've looked at an introduction to simple ANNs for time series analysis https://mc.ai/an-introduction-on-time-series-forecasting-with-simple-neura-networks-lstm/.

In the tutorial, the train dataset r2 score is 0.897 and the test dataset r2 score is 0.789. They end up with some pretty neat predictions (see graph below, also from the same tutorial).

Now my question is, for real world time series data, is there a consensus on what r2 scores make for an acceptable model?

enter image description here

Dave
  • 28,473
  • 4
  • 52
  • 104
James
  • 69
  • 3
  • Watch out for the fact that $R^2$ does not represent the proportion of variance explained (the usual interpretation) when the regression is nonlinear, like a neural network is. – Dave Aug 19 '20 at 13:40
  • @Dave So what does it represent in a nonlinear model (if it represents anything at all)? – James Aug 19 '20 at 16:26
  • It's kind of a proxy for $MSE$ or $SSE$. If $1-SSE/SSTot$ is large, we know that $SSE$ (and therefore $MSE$) is small. We could compare two models on any of the three and get that one model outperforms the other (same result no matter if $MSE$, $SSE$, or $R^2$ is used). – Dave Aug 19 '20 at 17:26

1 Answers1

0

No, there is no such consensus but common sense and relative success (compared to others in the literature). An $R^2$ of 0.8-0.9 is typically a good thing, but in some applications 0.6 can also be regarded as very good. It all depends on how hard the problem is.

I know that this is not your original question, but I wanted to give you and anyone reading this post a heads up: Sometimes plotting true vs real in time series problems can be misleading about how good the predictions are because if the time range is long enough, even using a very simple estimator, e.g. $\hat x_t=x_{t-1}$, might give the impression that the true and predicted values overlap very well, however it is just a visual illusion. In your plot, I see a similar time lag between true vs predicted.

gunes
  • 49,700
  • 3
  • 39
  • 75