2

In Stata, I am performing the following regression on a data set

regress sales price female black income hs age

This yielded

R-squared: .3208

Adjusted R-Squared: .2282

So let's now suppose I drop the variable age.

regress sales price female black income hs

This yielded

R-squared: .1661

Adjusted R-Squared: .0936

Thus, incorporating the variable age in the regression significantly changes R squared and adjusted R squared.

My Question

If I remove a variable from a regression, does the change in R squared reflect the amount of R squared contributed by that variable?

Community
  • 1
Stan Shunpike
  • 3,623
  • 2
  • 27
  • 36
  • Could you tell us what "contributed by" means to you? In some reasonable senses the answer is clear *yes* whereas in other senses it is *no*. – whuber Oct 12 '16 at 13:35
  • I think I took it to mean what Jake Westfall said in his answer. However, I wondered if there were other possible meanings besides additivity. I left it open ended in the hopes that someone might reply with a different interpretation besides additivity since its not very interesting. Rarely is it the case that all my variables would be uncorrelated. – Stan Shunpike Oct 12 '16 at 17:43
  • One standard meaning is that the change in r-squared for a given regressor $X$ reflects (in the usual least-squares linear-relationship sense) additional information about the response *that is not already accounted for* in all the other regressors. That obviously depends on the other regressors (except when all of them are orthogonal to $X$). In any case, despite what @Jake writes, that there is no requirement that all the other regressors be mutually orthogonal: that's irrelevant. – whuber Oct 12 '16 at 18:17
  • @whuber To make sure I understand, you're saying that what's relevant for a given predictor $X$ is whether $X$ is orthogonal to the subspace spanned by the other predictors, but whether those other predictors are orthogonal amongst themselves is irrelevant, right? – Jake Westfall Oct 12 '16 at 22:58
  • That's exactly right, @Jake. – whuber Oct 13 '16 at 16:30
  • @whuber It's true of course, but I'm not sure I see how it contradicts my answer? (Or maybe you're not claiming that it contradicts my answer?) We can only get the additive decomposition of the total $R^2$ as the sum of all the partial $R^2$ values if the condition that you mentioned holds for every $X$ -- but if that condition holds for every $X$, this implies that all of the predictors are uncorrelated, yes? – Jake Westfall Oct 13 '16 at 17:08
  • There is no contradiction to your answer, @Jake, but what you write in your answer isn't relevant to the question. The question does not concern decomposing $R^2$ additively for *all* variables: it concerns only the effect of *one specified variable* ("that variable"). – whuber Oct 13 '16 at 17:11
  • @whuber Ah, I see – Jake Westfall Oct 13 '16 at 17:35

1 Answers1

5

Only if all of the predictors in the model are completely uncorrelated.

In that ideal case it is possible to neatly decompose the total $R^2$ into a set of partial $R^2$ values for each predictor, such that the partial $R^2$ values all add up to exactly the total $R^2$. However, if there are any correlations among the predictors, then the partial $R^2$ values generally won't add up nicely to the total $R^2$. They could add up to more than the total $R^2$ or less than the total $R^2$ -- this latter case would indicate a pattern of suppression among the predictors.

Community
  • 1
Jake Westfall
  • 11,539
  • 2
  • 48
  • 96
  • Thanks very much. This was very clear. Just to clarify, this applies to both $R^2$ and the adjusted $R^2$? – Stan Shunpike Oct 12 '16 at 02:38
  • @StanShunpike No, this all applies to the classical, un-adjusted $R^2$. I'm not positive, but I *think* that the adjusted $R^2$ values will not generally add up nicely even when all the predictors are uncorrelated (although it should of course be pretty close as long as the sample size is not tiny). – Jake Westfall Oct 12 '16 at 02:51