15

This may be a basic question, but I was wondering why an $R$ value in a regression model can simply be squared to give a figure of explained variance?

I understand that $R$ coefficient can give the strength of a relationship, but I don't understand how simply squaring this value gives a measure of explained variance.

Any easy explanation of this?

Thanks very much for helping with this!

Macro
  • 40,561
  • 8
  • 143
  • 148
David
  • 149
  • 1
  • 3
  • Are you looking for something intuitive or more mathematical? Have you looked through some of the other questions on $R^2$ and correlation coefficients on this site? – cardinal May 09 '12 at 23:46
  • 1
    Two related questions are [here](http://stats.stackexchange.com/questions/7357/) and [here](http://stats.stackexchange.com/questions/26176), for example. If you play around with the equations there, you'll be able to derive the mathematical equivalence. But, neither are likely to be particularly helpful from an intuition standpoint. – cardinal May 09 '12 at 23:47
  • I see this the opposite way. It is R square that is defined as 1 -residual variance/total variance and then R is postive square root of that. It just happens that when we have simple linear regression R square reduces to the square of the correlation coefficient. – Michael R. Chernick May 10 '12 at 00:37
  • @Michael, you undoubtedly intended to say the *appropriately signed* square root rather than the *positive* one. – cardinal May 10 '12 at 10:28
  • @cardinal I think the terminology is that R is the positive square root. It is certainly true that the correlation coefficient can be either positive or negative. So R is the absolute value of the correlation coefficient. – Michael R. Chernick May 10 '12 at 12:17
  • @Michael: That is interesting. When I see $R$, I think naturally of "sample correlation coefficient", not of its absolute value. What software or literature uses this terminology? For example, all the linear regression theory textbooks I am familiar with use the notation in the way I've described and `R` (the software) does not report such a quantity. – cardinal May 10 '12 at 12:37
  • 1
    @cardinal, I have the same impression - $R$ (or $r$) refers to the sample correlation coefficient and would be surprised to see a widely used reference that uses that to refer to the absolute value of the sample correlation – Macro May 10 '12 at 12:39
  • My recollection is that most textbooks that cover linear regression talk about R-square and the correction coefficient or the sample estimate of the correlation coefficient and never specifically mention R. Usually I think they would say something like "In the case of simple linear regression R-square is the square of the correlation coefficient." That would not settle this issue as they don't specifically mention or define R. I have been doing statistics for a long time and don't clearly remember how I got this definition of R. – Michael R. Chernick May 10 '12 at 14:31
  • Since both cardinal and Macro have never heard it I will look for a source (that is not one of my own books) that mentions it. Keep in mind that I am talking about the symbol R and not the sample correlation coefficient that is often denoted r. – Michael R. Chernick May 10 '12 at 14:32
  • I have once asked a question regarding the relationship between $R$ and $R^2$. Maybe it'll help shed some light on your question: http://stats.stackexchange.com/questions/65960/how-to-get-from-r-to-r2-the-hard-way – Waldir Leoncio Dec 12 '13 at 18:34

1 Answers1

17

Hand-wavingly, the correlation $R$ can be thought of as a measure of the angle between two vectors, the dependent vector $Y$ and the independent vector $X$. If the angle between the vectors is $\theta$, the correlation $R$ is $\cos(\theta)$. The part of $Y$ that is explained by $X$ is of length $||Y||\cos(\theta)$ and is parallel to $X$ (or the projection of $Y$ on $X$). The part that is not explained is of length $||Y||\sin(\theta)$ and is orthogonal to $X$. In terms of variances, we have $$\sigma_Y^2 = \sigma_Y^2\cos^2(\theta) + \sigma_Y^2\sin^2(\theta)$$ where the first term on the right is the explained variance and the second the unexplained variance. The fraction that is explained is thus $R^2$, not $R$.

Dilip Sarwate
  • 41,202
  • 4
  • 94
  • 200
  • 2
    (+1) Not too much handwaving going on here really. The geometric viewpoint *is* the most intuitive, in my view. There is likely to be a high-quality open-source figure out there that depicts things precisely this way. – cardinal May 10 '12 at 10:25
  • (+1) I started to write up a direct derivation that ${\rm cor}(y,\hat{y})^2$ was equal to the usual definition of $R^2$ as a ratio of variances but, in doing so, I noticed it provided little/no intuition (and so it probably wouldn't be helpful to the original poster) - I think this does! – Macro May 10 '12 at 12:17
  • 1
    This doesn't answer the question but shows how R square is mentioned as the square of the correlation coefficient without any reference to R. So sources confirming or refuting my claim may be hard to find. This is from an article on the coefficient of determination in Wikipedia: – Michael R. Chernick May 10 '12 at 14:55
  • As squared correlation coefficient Similarly, after least squares regression with a constant+linear model (i.e., simple linear regression), R2 equals the square of the correlation coefficient between the observed and modeled (predicted) data values. – Michael R. Chernick May 10 '12 at 14:56
  • Under general conditions, an R2 value is sometimes calculated as the square of the correlation coefficient between the original and modeled data values. In this case, the value is not directly a measure of how good the modeled values are, but rather a measure of how good a predictor might be constructed from the modeled values (by creating a revised predictor of the form α + βƒi). According to Everitt (2002, p. 78), this usage is specifically the definition of the term "coefficient of determination": the square of the correlation between two (general) variables. – Michael R. Chernick May 10 '12 at 14:56
  • @cardinal "There is likely to be a high-quality open-source figure out there that depicts things precisely this way" - I made an attempt at such a diagram in my answer to [Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$](http://stats.stackexchange.com/questions/123651/geometric-interpretation-of-multiple-correlation-coefficient-r-and-coefficient) – Silverfish Feb 03 '15 at 02:34