9

I am interested in stating that ___ % of the variance in Y is explained uniquely by $X_1$ and ___ % is explained uniquely by $X_2$.

  • Is there some way to obtain this from a multiple regression model, or do I need to obtain adjusted $R^2$ values from a series of residual regressions (sensu Legendre & Legendre et al.)?
  • That being asked, is $R^2$ value valid for insignificant OLS regression model?
  • Does the value of $R^2$ depend on a statistical test?
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Patrick
  • 1,381
  • 1
  • 15
  • 21
  • 4
    To respond to the parts of your question not covered in the answer - the value of $R^2$ does not depend on significance of any predictors or on whether you tested them. – Glen_b May 11 '13 at 23:00
  • Thanks Glen_b, OK, so I can perform partial regressions to estimate % of the variance in Y is explained uniquely by X1 and by X2 (using the adjusted R^2 value), even if coefficient for X2|X1 is not significant. I have necessarily collinear regressor variables (.5 < |r| < .6) that have a theoretical link to Y. I unfortunately also have small sample sizes (n ~ 20), but would like to use this to more or less determine if one of these regressors may influence Y to a greater degree, while statistically controlling for the other. Do you have any opinion to the validity of this approach? – Patrick May 11 '13 at 23:22
  • I'm not overly concerned with building predictive models, I simply want to know if evidence suggests X1|X2 is more influential than X2|X1 ... – Patrick May 11 '13 at 23:24
  • 2
    The part about estimating % of variance uniquely explained is, I think, covered in the answer you have already. – Glen_b May 11 '13 at 23:43
  • It absolutely is, thank you. Are you aware of any literature regarding either what you have stated above, or regarding this approach with the use with small sample sizes? If so, that would be greatly appreciated. Thanks again. – Patrick May 11 '13 at 23:55
  • squared semi-partial correlation is what I've seen most people use for this. However, I was curious [whether it is an unbiased estimator](http://stats.stackexchange.com/questions/58758/unbiased-estimate-of-the-semi-partial-correlation) – Jeromy Anglim May 12 '13 at 03:04
  • I would like to add that it has been shown that estimates of $R^2$ are unreliable when the number of observations per parameter in the model is low. See Austin, P. C. and Steyerberg, E. W. (2015). The number of subjects per variable required in linear regression analyses. *Journal of clinical epidemiology*, 68(6): 627-636. – Phil Oct 29 '18 at 09:17

2 Answers2

4

Yes, you're trying to calculate the Extra Sum of Squares. In short you are partitioning the regression sum of squares. Assume we have two $X$ variables, $X_1$ and $X_2$. The $SSTO$ (total sum of squares, made up of the SSR and SSE) is the same regardless of how many $X$ variables we have. Denote the $SSR$ and $SSE$ to indicate which $X$ variables are in the model: e.g.

$SSR(X_1,X_2) = 385$ and $SSE(X_1,X_2) = 110$

Now let's assume we did the regression just on $X_1$ e.g.

$SSR(X_1) = 352$ and $SSE(X_1) = 143$.

The (marginal) increase in the regression sum of squares in $X_2$ given that $X_1$ is already in the model is:

\begin{eqnarray} SSR(X_2|X_1)& = &SSR(X_1,X_2) - SSR(X_1)\\ & = & 385 - 352\\ & = & 33 \end{eqnarray}

or equivalently, the extra reduction in the error sum of squares associated with $X_2$ given that $X_1$ is already in the model is:

\begin{eqnarray} SSR(X_2|X_1) & = & SSE(X_1) - SSE(X_2,X_1)\\ &=& 143 - 110\\ &=& 33 \end{eqnarray}

In the same way we can find:

\begin{eqnarray} SSR(X_1|X_2) &=& SSE(X_2) - SSE(X_1,X_2)\\ &=& SSR(X_1,X_2) - SSR(X_2) \end{eqnarray}

Of course, this also works for more $X$ variables as well.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
Eric Peterson
  • 2,323
  • 14
  • 20
1
  • I think you have to use multiple regressions as indicated by Eric Peterson. The other option is that you use partial correlation coefficients. That would make sense, if the absolute values of $y$ and the magnitude of the squares are not important.
  • In a certain sense, $R^2$ is very valid for an insignificant OLS model. The significance thresholds are after all lines drawn in water. For example, if you have to choose between models to use for making predictions, it makes sense to use the model with highest $R^2$. Or, if the models involve a different number of covariates, better use adjusted $R^2$ denoted as $\bar{R}^2$.
  • $R^2$ is actually the square of the correlation coefficient between predicted values and the reality, i.e. $R^2=(cor(y_i,\hat{y}_i))^2$, and thus, you can use tables for the correlation coefficient to test the ovarall significance of your model. The other option is that you use the F test.
stats_boy
  • 46
  • 3