2

Let's take a model like this:

mod <- lm(Petal.Width ~ Species, data = iris)

Is there any way to know how much variance is explained by each level of Species overall? By overall I mean NOT just relative to a reference level.

I know that emmeans can compare coefficients of each level of a categorical predictor to the overall mean. Is something like this possible for amount of variance explained?

Dave
  • 1,641
  • 2
  • 14
  • 27

2 Answers2

2

The difficulty here is that a level of a categrical variable is not a variable --- it is an outcome of the variable. So this is a bit like asking if it is possible to get a measure of variance explained for a continuous variable for the outcome $X=3$.

While it is an odd question, I suppose that the answer is yes, it is possible (though it would no longer be the same model). What you would do is to remove the initial categorical/continuous variable and replace it with a single indicator variable for the outcome of interest. You can then use ANOVA to measure the variance explained by this indicator variable. This would still compare the outcome to a reference level, but the reference level would be all cases where that outcome does not occur. Of course, this is not the same model as you are using when you include the full categorical variable (which I'm assuming has ore than two outcomes), but that would be the appropriate comparison.

Ben
  • 91,027
  • 3
  • 150
  • 376
  • Thanks! So just to be sure I have this correct, I would essentially run a model that codes species setosa = 1 and other levels of species = 0. That would give me variance explained for setosa. I would then do this for each level? (In reality, I think that the reviewer asking for this has overlooked the complexities of this request.) – Dave Jan 13 '22 at 22:15
  • 1
    Well, remember that variance-explained is measured conditionally on what terms are already in the model. So really, it depends on what conditional variance-explained you actually want to measure. If you do comparisons seperately for each level then you will get a bunch of measures of the variance explained for each category, but each is conditional on no other category being in the model (so you can't add them up). – Ben Jan 14 '22 at 02:04
0

Here's one idea, which may very well match Ben's answer. First, obtain the "eff" contrasts, which comprise comparisons between each mean and the average of all the means:

> mod <- lm(Petal.Width ~ Species, data = iris)
> library(emmeans)
> EMM <- emmeans(mod, "Species")
> (CON = contrast(EMM, "eff"))
 contrast          estimate     SE  df t.ratio p.value
 setosa effect       -0.953 0.0236 147 -40.343  <.0001
 versicolor effect    0.127 0.0236 147   5.360  <.0001
 virginica effect     0.827 0.0236 147  34.982  <.0001

P value adjustment: fdr method for 3 tests

Now, there is a relation between $R^2$ and $F$: $$ R^2 = 1 - (1 + df_R\cdot F / df_E)^{-1} $$ where $df_R$ is the d.f. for regression and $df_E$ is the d.f. for error. In this case, we want to develop an $R^2$-like statistic for each level of Species. And note that each $t^2$ is an $F$ statistic with one numerator d.f. accordingly, calculate:

> F <- test(CON)$t.ratio^2
> 1 - 1 / (1 + F/147)
[1] 0.9171609 0.1634979 0.8927607

I'll claim these are like $R^2$ statistics. They sum to more than 1 because they are interdependent.

Russ Lenth
  • 15,161
  • 20
  • 53
  • Thank you! Is there anything I can cite for this? – Dave Jan 17 '22 at 20:36
  • Nope, just an idea and maybe not a good one. Many regression texts give the same relation as the one I linked in CV. BTW,if you do this with`MAX = contrast(EMM, list (c = test(CON)$estimate))`, you wind up with the R^2 for the model. That is a result of the derivation of the Scheffe method. – Russ Lenth Jan 17 '22 at 21:10
  • Cool, thanks again for the idea! – Dave Jan 17 '22 at 22:26