Does $r$-squared have a $p$-value?

Question

I seem to have confused myself trying to understand if a $r$-squared value also has a $p$-value.

As I understand it, in linear correlation with a set of data points $r$ can have a value ranging from $-1$ to $1$ and this value, whatever it is, can have a $p$-value which shows if $r$ is significantly different from $0$ (i.e., if there is a linear correlation between the two variables).

Moving on to linear regression, a function can be fitted to the data, described by the equation $Y = a + bX$. $a$ and $b$ (intercept and slope) also have $p$-values to show if they are significantly different from $0$.

Assuming I so far have understood everything correct, are the $p$-value for $r$ and the $p$-value for $b$ just the same thing? Is it then correct to say that it is not $r$-squared that has a $p$-value but rather $r$ or $b$ that does?

Answer 1

In addition to the numerous (correct) comments by other users pointing out that the $p$-value for $r^2$ is identical to the $p$-value for the global $F$ test, note that you can also get the $p$-value associated with $r^2$ "directly" using the fact that $r^2$ under the null hypothesis is distributed as $\textrm{Beta}(\frac{v_n}{2},\frac{v_d}{2})$, where $v_n$ and $v_d$ are the numerator and denominator degrees of freedom, respectively, for the associated $F$-statistic.

The 3rd bullet point in the Derived from other distributions subsection of the Wikipedia entry on the beta distribution tells us that:

If $X \sim \chi^2(\alpha)$ and $Y \sim \chi^2(\beta)$ are independent, then $\frac{X}{X+Y} \sim \textrm{Beta}(\frac{\alpha}{2}, \frac{\beta}{2})$.

Well, we can write $r^2$ in that $\frac{X}{X+Y}$ form.

Let $SS_Y$ be the total sum of squares for a variable $Y$, $SS_E$ be the sum of squared errors for a regression of $Y$ on some other variables, and $SS_R$ be the "sum of squares reduced," that is, $SS_R=SS_Y-SS_E$. Then $$ r^2=1-\frac{SS_E}{SS_Y}=\frac{SS_Y-SS_E}{SS_Y}=\frac{SS_R}{SS_R+SS_E} $$ And of course, being sums of squares, $SS_R$ and $SS_E$ are both distributed as $\chi^2$ with $v_n$ and $v_d$ degrees of freedom, respectively. Therefore, $$ r^2 \sim \textrm{Beta}(\frac{v_n}{2},\frac{v_d}{2}) $$ (Of course, I didn't show that the two chi-squares are independent. Maybe a commentator can say something about that.)

Demonstration in R (borrowing code from @gung):

set.seed(111)
x = runif(20)
y = 5 + rnorm(20)
cor.test(x,y)

# Pearson's product-moment correlation
# 
# data:  x and y
# t = 1.151, df = 18, p-value = 0.2648
# alternative hypothesis: true correlation is not equal to 0
# 95 percent confidence interval:
#  -0.2043606  0.6312210
# sample estimates:
#       cor 
# 0.2618393 

summary(lm(y~x))

# Call:
#   lm(formula = y ~ x)
# 
# Residuals:
#     Min      1Q  Median      3Q     Max 
# -1.6399 -0.6246  0.1968  0.5168  2.0355 
# 
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)    
# (Intercept)   4.6077     0.4534  10.163 6.96e-09 ***
# x             1.1121     0.9662   1.151    0.265    
# ---
#   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# 
# Residual standard error: 1.061 on 18 degrees of freedom
# Multiple R-squared:  0.06856,  Adjusted R-squared:  0.01681 
# F-statistic: 1.325 on 1 and 18 DF,  p-value: 0.2648

1 - pbeta(0.06856, 1/2, 18/2)

# [1] 0.2647731

Answer 2

I hope this fourth (!) answer clarifies things further.

In simple linear regression, there are three equivalent tests:

1. t-test for zero population slope of covariable $X$
2. t-test for zero population correlation between $X$ and response $Y$
3. F-test for zero population R-squared, i.e. nothing of the variability of $Y$ can be explained by differing $X$.

All three tests check for a linear association between $X$ and $Y$ and, fortunately(!), they all lead to the same result. Their test statistics are equivalent. (Tests 1 & 2 are based on the Student-distribution with $n-2$ df which corresponds to the sampling F-distribution of test 3, just with squared test statistic).

A quick example in R:

# Input
set.seed(3)

n <- 100
X <- runif(n)
Y <- rnorm(n) + X

cor.test(~ X + Y) # For test 2 (correlation)

# Output (part)
# t = 3.1472, df = 98, p-value = 0.002184
# alternative hypothesis: true correlation is not equal to 0

# Input (for the other two tests)
fit <- lm(Y ~ X)
summary(fit)      

# Output (partial)
Coefficients:
            Estimate Std. Error t value Pr(>|t|)   
(Intercept) -0.03173    0.18214  -0.174  0.86204   
X            1.02051    0.32426   3.147  0.00218 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9239 on 98 degrees of freedom
Multiple R-squared:  0.09179,   Adjusted R-squared:  0.08253 
F-statistic: 9.905 on 1 and 98 DF,  p-value: 0.002184

As you can see, the three tests yield the same p value of 0.00218. Note that test 3 is the one in the last line of the output.

So your F-test for the R-squared is a very frequent one, although not many statisticians are interpreting it as a test for the R-squared.

Answer 3

You seem to have a decent understanding to me. We could get a $p$-value for $r^2$, but since it is a (non-stochastic) function of $r$, the $p$s would be identical.

Answer 4

There are several ways of deriving the test statistic for tests of the Pearson correlation, $\rho$. To obtain a $p$-value, it is worth emphasizing that you need both a test and a sampling distribution of a test statistic under the null hypothesis. Your title and question seems to have some confusion between Pearson correlation and the "variance explained" $r^2$. I will consider the correlation coefficient first.

There is no "best" way to test the Pearson correlation which I'm aware of. Fisher's Z transformation is one such way, based on hyperbolic transformations, so that the inference is a little bit more efficient. This is certainly a "good" approach, but the sad bit is that inference for this parameter is consistent with inference about the slope parameter $\beta$ for association: they tell the same story in the long run.

The reason why statisticians have (classically) wholly preferred tests of $\beta$ is because we do have a "best" test: linear regression, which is the BLUE estimator. In the days of modern statistics, we don't really care if a test is "best" any more, but linear regression has plenty of other fantastic properties that justify its continued usage for determining the association between two variables. In general, your intuition is right: they're essentially the same thing, and we focus our attention upon $\beta$ as a more practical measure of association.

The $r^2$ is a function of both the slope and the intercept. If either of these values are nonzero, the $r^2$ should have a discernable sampling distribution relative to that which would be expected if the linear parameters were zero. However, deriving distributions of $r^2$ under the null and comparing to $r^2$ under some alternative hypothesis doesn't give me much confidence that this test has much power to detect what we want it to. Just a gut feeling. Again turning to "best" estimators, OLS gives us "best" estimates of both the slope and the intercept, so we have that confidence that our test is at least good for determining the same (if any) association by directly testing the model parameters. To me, jointly testing the $\alpha$ and $\beta$ with OLS is superior to any test about $r^2$ except in a rare case of (perhaps) a non-nested predictive modeling calibration application... but BIC would probably be a better measure in that scenario anyway.

Answer 5

This isn't quite how I would interpret things. I don't think I'd ever calculate a $p$-value for $r$ or $r^2$. $r$ and $r^2$ are qualitative measures of a model, not measures that we're comparing to a distribution, so a $p$-value doesn't really make sense.

Getting a $p$-value for $b$ makes a lot of sense - that's what tells you whether the model has a linear relationship or not. If $b$ is statistically significantly different from $0$ then you conclude that there is a linear relationship between the variables. The $r$ or $r^2$ then tells you how well the model explains the variation in the data. If $r^2$ is low, then your independent variable isn't helping to explain very much about the dependent variable.

A $p$-value for $a$ tells us if the intercept is statistically significantly different from $0$ or not. This is of varying usefulness, depending on the data. My favorite example: if you do a linear regression between gestation time and birth weight you might find an intercept of, say, 8 ounces that is statistically different from $0$. However, since the intercept represents a gestation age of $0$ weeks, it doesn't really mean anything.

If anyone does regularly calculate $p$-values for an $r^2$ I'd be interested in hearing about them.