RMSE to accuracy

Question

I have seen multiple questions and answers about this, but I haven't been able to understand, so, I'm gonna try to ask as simple as possible. I have built several models to forecast future value of a variable. When I test the model (on out of training data) I have received the RMSE obtained by each one.

Example:

RMSE: 0.05, Min value of dataset: 1, Max value of dataset: 2

How can I calculate the accuracy of the model? Sorry if the question is duplicated, thanks in advance.

Is this a classification or regression problem? – Dave Sep 08 '20 at 23:22 — Dave, Sep 08 '20 at 23:22
Then what do you mean by “accuracy”? – Dave Sep 08 '20 at 23:23 — Dave, Sep 08 '20 at 23:23
The effectiveness (in percentage), is it wrong? – David Díaz Sep 08 '20 at 23:26 — David Díaz, Sep 08 '20 at 23:26

score 1 · Accepted Answer · answered Sep 10 '20 at 11:17

Answering with a question: you would like to calculate "percentage" of what exactly? If you want to have a unitless measure, you can take a look at MAPE (see mape), or $R^2$ (see r-squared), but as you can learn from What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? and Is $R^2$ useful or dangerous?, both have their shortcomings. Moreover, accuracy is a far from perfect measure of model performance, and there are many issues with it, so for classification problems this is often not the best choice for a metric as well.

TL;DR from the threads mentioned above is that for each "unitless" metric, you need to normalize it somehow and often how you normalize will influence the result and may lead to misguided interpretations.

For regression problems it is usually much more useful to have metrics using units of the thing that you are predicting, like mean absolute error, or root mean squared error, e.g. if you are predicting prices, they tell you by how many euros you are off, on average, etc.

RMSE to accuracy

1 Answers1