1

Okay so I think I found a formula for the coefficient estimates but it is not very concise. It has like 6 sum of squares but it is in a single fraction so it is calculable. I was wondering what the simplest formula would be for estimating the coefficients for a linear regression. So say you had the regression equation: $$y = b_0 + b_1 (x-\overline{x}) + b_2 (z-\overline{z}) + \epsilon$$ where $\overline{x}$ and $\overline{z}$ are averages of $x$ and $z$. So (with no use of matrices) what is the simplest formula you can get that involves the first and second sample moments of $y$, $x$ and $z$ for the coefficient estimates ($\hat{b}_1$ and $\hat{b}_2$)?

I just minimized the residual squared and used $s_{xx}$, $s_{xz}$ etc. notation to find the formulas but as I said they are not very nice.

Comp_Warrior
  • 2,075
  • 1
  • 20
  • 35
Mauro Augusto
  • 489
  • 6
  • 13
  • I know you say that you do not want to use matrices, but have you tried solving the matrix equations? If you do, you should arrive at the conclusion you are looking for. –  Sep 15 '13 at 17:08
  • @BabakP it is not that I do not want to, but my university stopped using matrices for a while now so our courses do not teach it. – Mauro Augusto Sep 15 '13 at 18:06
  • If that is the case, will you accept an answer with matrices? –  Sep 15 '13 at 18:30
  • @BabakP I am sorry but that is completely useless to me, I do not understand it... (I have not learned how to use matrices to that extent in the field of econometrics). Thanks for trying though. – Mauro Augusto Sep 15 '13 at 19:47
  • You can obtain formulas without using matrix arithmetic by using repeated "linear matching" as described at http://stats.stackexchange.com/a/46508. Better, though, would be to learn how to work with matrices: there are plenty of textbooks and online resources. Your education should not be limited to what your university offers! (I find it unbelievable, though, that your university does not teach this material, it is so basic. You might want to do more research into what is available where you are.) – whuber Nov 15 '13 at 19:41

2 Answers2

1

Are these the "not very nice" formulas that you found?
(Note that $\hat{b}_1$ and $\hat{b}_2$ have the same denominator.)

$\hat{b}_1 = (s_{zz}\,s_{xy} - s_{xz}\,s_{zy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$

$\hat{b}_2 = (s_{xx}\,s_{zy} - s_{xz}\,s_{xy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$

$\hat{b}_0 = \overline{y}$

Ray Koopman
  • 2,143
  • 12
  • 6
0

Consider rewriting the equation into matrix form. To make notation simpler, I will define $x_i^*=x_i-\bar x$ and $z_i^*=z_i-\bar z$. Now we can rewrite $$y_i = b_0 + b_1 (x_i-\overline{x}) + b_2 (z_i-\overline{z}) + \epsilon_i$$ as the following:

$$\mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{\epsilon}$$ Now to obtain coefficient estimates for $b_1$ and $b_2$ we can solve the the least square estimator $$\hat\beta=(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$$ If it help to see, we can explicitly write the matrix equation $$\mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{\epsilon}$$ as $$\begin{pmatrix} y_1\\ y_2\\ \vdots\\ y_n \end{pmatrix}=\begin{pmatrix} 1&x_1^*&z_1^*\\ 1&x_2^*&z_2^*\\ \vdots&\vdots&\vdots\\ 1&x_n^*&z_n^*\\ \end{pmatrix}\begin{pmatrix} b_0\\ b_1\\ b_2 \end{pmatrix}+\begin{pmatrix} \epsilon_1\\ \epsilon_2\\ \vdots\\ \epsilon_n\\ \end{pmatrix}$$