5

This blog post has a nice description of when the square of the Pearson correlation coefficient, r, is equal to the coefficient of determination, $R^2$. Specifically, states that they will be the same when the model, $f$, meets these three conditions:

  1. $f$ is the model that minimizes squared-error loss
  2. Because it is the optimum (in the sense of item 1), there is no shift of $f$ that will improve the fit.
  3. Because it is the optimum (in the sense of item 1), there is no scaling of $f$ that will improve the fit.

Surely the last two conditions aren't redundant, but I can't think of an instance where $f$ is optimal in the least squares sense, but improvable by shifting or scaling. Can someone explain how that might happen?

naught101
  • 4,973
  • 1
  • 51
  • 85
  • 1
    1. The blog post you link to doesn't seem to actually say what you say it says - as far as I can see it doesn't say $r=R^2$ when those conditions hold (and if it does say $r=R^2$ somewhere, it would be flat out wrong). 2. Unless I misread it, it also seems to miss that since $R^2$ explicitly *is* the square of a correlation it necessarily shares all the properties of a squared correlation. – Glen_b Mar 19 '15 at 00:51
  • 1
    To be more explicit, if you have only a single $x$, then $R^2=\text{corr}(y,x)^2=r^2$ (and for multiple predictors, $R^2=\text{corr}(y,\hat{y})^2$). For a single predictor $R^2=r^2$ always. Of course $r^2=r$ only when $r=0$ or $1$. At best I find the post terribly unclear, and I hope that I'm just misunderstanding what's being got at. Nina Zumel is certainly not stupid, but I don't think this is actually a good post; it seems to take a rather odd approach to well-established ideas. Maybe something was lost in translation. – Glen_b Mar 19 '15 at 01:00
  • 1
    The way to "get from $R^2$ to correlation" is to take the square root of it ($|r|=\sqrt{R^2}$) and attach the sign of the coefficient in the regression. – Glen_b Mar 19 '15 at 01:11
  • @Glen_b: sorry, the question title got lost, and I re-typed it without thinking. Fixed. – naught101 Mar 19 '15 at 01:15
  • 4
    There is a related question [Is there any difference between $r^2$ and $R^2$?](http://stats.stackexchange.com/questions/134167/is-there-any-difference-between-r2-and-r2) - it's worth pointing out that in regressions without intercept terms not only does the basic relationship stop holding, but to complicate matters more, people (and software) often use an alternative definition of $R^2$ anyway. – Silverfish Mar 19 '15 at 01:35
  • 1
    Okay, thanks. The question makes some more sense now. I still think I'm missing something important about what the linked post is getting at - what's taken as a given and what's not. In a sense it's not clear what the circumstances are. If we're talking about simple linear regression, $r^2=R^2$, end of story. So we mustn't be talking about just that, but some broader conception -- in which case what *are* we talking about, exactly? What are the conditions we're dealing with? – Glen_b Mar 19 '15 at 01:37
  • @Glen_b: Yes, I guess that's the point of my question. What kind of model, $f$, fits the first condition, but does not fit one or both of the second two? – naught101 Mar 19 '15 at 02:31

1 Answers1

2

Partially answered in comments, summarized below.

If you have only a single $x$, then $R^2=\operatorname{corr}(y,x)^2=r^2$ (and for multiple predictors, $R^2=\operatorname{corr}(y,\hat{y})^2$). For a single predictor $R^2=r^2$ always. Of course $r^2=r$ only when $r=0$ or $1$. At best I find the post terribly unclear, and I hope that I'm just misunderstanding what's being got at. Nina Zumel is certainly not stupid, but I don't think this is actually a good post; it seems to take a rather odd approach to well-established ideas. Maybe something was lost in translation. – Glen_b (source comment)

There is a related question Is there any difference between $r^2$ and $R^2$? - it's worth pointing out that in regressions without intercept terms not only does the basic relationship stop holding, but to complicate matters more, people (and software) often use an alternative definition of R2 anyway. – Silverfish (source comment)

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467