2

In package relaimpo in R, we measure the relative importance with different methods. However, the $R^2$ for all the methods is similar.

Is it supposed to be similar or a bug?

Nick
  • 792
  • 5
  • 25
MFR
  • 552
  • 5
  • 15
  • What do you measure? For what methods? What is your data, what models do you compare? – Tim Sep 13 '16 at 07:48
  • I measure the relative importance of some variables. Relaimpo package has a range of methods which are c("lmg", "pmvd", "last", "first", "betasq", "pratt", "genizi", "car"). I hoped that using different methods give me different coefficient of determination but it is not true. Wondering if I'm missing something or it is a bug – MFR Sep 13 '16 at 07:53

1 Answers1

2

$R^2$ measures fit of the model, not of particular variables.

From the documentation (bold added by myself):

Five of the metrics in calc.relimp (lmg, pmvd, pratt, genizi and car), decompose the model $R^2$. calc.relimp (lmg, pmvd, pratt, genizi and car) sum to the $R^2$ that is to be decomposed [...]

You can find out more about the methods from the references provided in the documentation.

Notice however that $R^2$ can be misleading and does not really tell you about "variance explained" unless you are using linear regression.

Community
  • 1
Tim
  • 108,699
  • 20
  • 212
  • 390
  • Thanks for the quick reply, do you have any idea if there is a way to compare theses methods? In my case, there are different weights in different methods? How do I know which one is more accurate? – MFR Sep 13 '16 at 08:07
  • What models? You use "methods", "models" and "variables" interchangeably what makes it unclear what do you mean. Do you compare different models? What models? – Tim Sep 13 '16 at 08:10
  • @Eddie: The different importance metrics each purport to replace our fuzzy, qualitative notion of "importance" with something exact & quantitative. The question of their accuracy doesn't arise. – Scortchi - Reinstate Monica Sep 13 '16 at 08:16
  • Thanks Scortchi, It's making more sense to me now. How people know which metrics they need to use in practice? Is it anyway to make these metrics actionable? – MFR Sep 13 '16 at 22:49