-1

Context: The linear regression Y = mX + c

Intuitively, it seems to me that swapping the independent variable and dependent variables won't change the test statistic for the significance of the coefficient for the dependent variable (the null hypothesis is that the coefficient is zero).

i.e. for the same $X$ and $Y$, $$Y = \hat{\beta_0} + \hat{\beta_1} X\\ X = \hat{\beta_0}' + \hat{\beta_1}' Y$$

is it true that, $$ \hat{\beta_1} /\mathrm{SE}(\hat{\beta_1} ) = \hat{\beta_1}'/\mathrm{SE}(\hat{\beta_1}')$$

Such that either way I do the hypothesis test, $$\mathrm{H}_0 : \beta_1 = 0 \\ \mathrm{H}_1 : \beta_1 \neq 0 \\ \mathrm{H}_0': \beta_1' = 0 \\ \mathrm{H}_1': \beta_1' \neq 0 $$

I will reject/accept with the same $p$-value, and obtain the same $R^2$ for the model?

I've tested this out on a data set that is available to me, and it seems to be true, but I'm not sure about the mathematics of it.

ning
  • 139
  • 6
  • "the test statistic for the significance of the coefficient for the dependent variable" what is difference between test statistic and significance ? these are different or same ? –  Apr 20 '18 at 14:26

1 Answers1

2

Galton (1886) showed that, if your variables are standardized (so that they have a unit variance), by reversing DV and IV you get the same coefficient. The reason for that is that: $\beta_1=\frac{Cov(X,Y)}{Var(X)}$;
$\beta_1'= \frac{Cov(X,Y)}{Var(Y)}$. Basically, you are always testing the null hypothesis of null correlation (given $Corr(X,Y)=0 \iff Cov(X,Y)=0).$

So, the answer is Yes: you will reject/accept with the same p-value, and obtain the same R^2 for the model.

Galton, Francis (1886): “Regression Towards Mediocrity in Hereditary Stature,” The Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246-263.

Federico Tedeschi
  • 814
  • 5
  • 17
  • Basically, you are always testing the null hypothesis of null correlation ? please elaborate ? –  Apr 20 '18 at 14:35
  • $Cov(X,Y)/Var(X)=0 \iff Cov(X,Y)/Var(Y)=0 \iff Cov(X,Y)=0$ given a ratio is null $\iff$ its numerator is null. Then, $Cov(X,Y)=0$. But since $Cov(X,Y)=\rho(X,Y)*\sigma_X*\sigma_Y$, $Cov(X,Y)=0 \iff \rho(X,Y)=0$. Of course I'm not considering here degenerate cases where either $X$ or $Y$ do not vary (a regression only makes sense when there is variation). – Federico Tedeschi Apr 20 '18 at 15:07