3

There is a well-answered question here.

But unfortunately, I don't even understand how the first equation in the answer is derived. Could someone help explain that?

$$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$

What is $r$ here?

Community
  • 1
  • 3
    $r_{ab}$ is the correlation between $a$ and $b$. – Glen_b Jun 08 '16 at 08:44
  • Thanks @Glen_b but I am still not sure I understand this equation. I think this $\beta$ is the same as $(X_1^TX_1)^{-1}X_1^TY$, how is this related with the right-hand-side? –  Jun 08 '16 at 18:34
  • Thanks. @Glen_b I found it [here](http://www.personality-project.org/r/book/chapter5.pdf) –  Jun 08 '16 at 19:00
  • 2
    `how is this related with the right-hand-side?` First, consider the formula (the 1st one [here](http://stats.stackexchange.com/a/44290/3277) which is homologic with the right-hand one but is for $b$, not for $\beta$. Both this and that are found in nearly any book on linear regression. Second, considering X having just 2 columns, "unwrap" algebraically and in scalar (in place of matrix) notation the expression $(X^TX)^{-1}X^TY$; and I'm sure you will arrive exactly at the formula for $b$. – ttnphns Jun 08 '16 at 20:51

1 Answers1

2

As the comment points out, $r_{ab}$ is the correlation between $a$ and $b$.

This equation: $$\text{Beta:} \quad \beta_{x_1} = \frac{r_{yx_1} - r_{yx_2}r_{x_1x_2} }{1-r_{x_1x_2}^2}$$ can be achieved by solving the following equation:

$$ \beta_{x_1} + r_{x_1x_2}\beta_{x_2} = r_{x_1y} \\ r_{x_1x_2}\beta_{x_1} + \beta_{x_2} = r_{x_2y} $$

where $x_1$ and $x_2$ are two predictors and $y$ is the dependent variable. Solving this above equation set by basic linear algebra will lead to the first equation.

Details can be found in Chapter 5 Multiple correlation and multiple regression of An introduction to psychometric theory with applications in R by William Revelle.