0

I know that an R² score of 1 is a perfect fit of the model to the truth, a 0 is an constant output regardless of the input, and that negative values are possible when the output varies, but there is no correlation with the input. (At least that's how I understand it?)

But... how do I interpret these values? Or: how do I compare two values with eachother? Comparing 0.8 with 0.9 is obvious: the 2nd value is better than the first. But how to compare low values with negative values?

For example, I have this graph:

Prediction vs truth

The green curve is the measurement, while the blue curve is the prediction. I'd expect the R² score to be very low, because it clearly isn't a good prediction, but... turns out, it is a negative value: -1.19.

How do I know which is better? A score of 0.19? Or a score of -1.19?

considering a score of 0 means a constant (wrong) value, then -1.19 must be better than 0?

Follow-up Question: I'm working on a project where I compare several techniques by their R²-score. I was planning on drawing a bar-plot with all the results as an overview. How would I do this, when -1.19 is better than 0, but 1 is best?

Follow-up Question 2: Maybe I'm using the wrong type of metric for this kind of comparison? Am I?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Opifex
  • 101
  • 2
  • 1
    I use R-squared (R2) to tell me how much of the data variance is explained by the regression model, calculated as "R2 = 1.0 - (regression_error_variance / dependent_data_variance)" . If the model fits perfectly through all data points and that means all errors are zero, and the error variance is zero because all errors are equal value - in a perfect fit R2 = 1.0 - (0.0 / dependent_data_variance) or just 1.0. Using this calculation, R-squared can only be negative if the error variance is *greater* than the dependent data variance which would tell me that the model is horribly bad. – James Phillips Apr 12 '19 at 21:28
  • Also relevant: https://stats.stackexchange.com/questions/13314/is-r2-useful-or-dangerous and https://stats.stackexchange.com/questions/414349/is-my-model-any-good-based-on-the-diagnostic-metric-r2-auc-accuracy-e – mkt Aug 16 '19 at 19:29

1 Answers1

1

Here's an image where different R2 Scores are compared:

Compare R2

The blue dots are the ground truth data. Each line has a different prediction. As expected, note that the orange line has an R2 score very close to 1. Also note that the red fits very badly the dataset and has a negative score of -1.55. The intuition here is that a negative score means that the model is in the opposite trend of the data.

In case you are familiar with Python, feel free to play with the source code.

Fernando Wittmann
  • 491
  • 4
  • 5
  • I don't get why you got downvoted. Even though this question is very old, it is exactly what I was asking back then. – Opifex Aug 16 '19 at 20:22
  • Note to downvoters: This answer is meant for those coming from Google since it is being ranked as the first result when searching for 'comparing r2 scores' – Fernando Wittmann Feb 04 '21 at 18:55