Finding the Coefficients of regressors

Question

I understand how to find the coefficients of a bivariate regression and univariate regression w/o an intercept, i.e:

Univariate: Y = BX + e

Bols = 
 = (X'X)^(-1) X'Y 
 = (X'X)^(-1) X'(BX + e)
 = B + (X'X)^(-1) Xe                      If we take expectation/ e and X are independent 
 = B

Bivariate: Y = X1*C + X2*B + e

X1*M_X2*Y = X1*M_X2*X1*C + X1*M_X2*X2*B + X1*M_X2*e
X1*M_X2*Y = X1*M_X2*X1*C + X1*M_X2*e      Assuming X and e are independent
C =  (X1*M_X2*Y)* (X1*M_X2*X1*C)^(-1) 
B =  (X2*M_X1*Y)* (X2*M_X1*X2*C)^(-1)

But when there is an intercept I am pretty confused about what you are supposed to do when an intercept is included.

I started to do univariate and got:

Univariate: Y = a + Bx + e

Bols = 
 = (X'X)^(-1) X'Y 
 = (X'X)^(-1) X'(a + BX + e)
 = (X'X)^(-1) X'(a) + B + (X'X)^(-1)X'e  Take E, E[e|x] = 0                      
 = (X'X)^(-1) X'(a) + B 
   Bols is a vector: [a  B] [ E[X'X]^(-1) ]*E[X']   1]'

But I am not sure if this is correct?

Or what should be done if it is bivariate: Y= a + Cx1 + Bx2 + e ?

Thanks!

I answered this question in detail last night on a closely related thread at http://stats.stackexchange.com/questions/196807/explicit-solution-for-linear-regression-with-two-predictors/197788#197788. Although they would be hard to search for, there are several dozen other posts that show how to construct the design matrix $X$--you could probably find some by exploring our [tag:regression]-related questions. — whuber, Feb 22 '16 at 14:14

score 2 · Accepted Answer · answered Feb 22 '16 at 09:13

Actually, you already give all relevant formulae yourself. Suppose you have for $N$ individuals a regression $Y_i = a + cx_{1,i} + bx_{2,i} + e_i, \; 1\leq i \leq N$. Now rewrite $x_i = (1, x_{1,i}, x_{2,i})'$ and stack these $x_i's$ in $X$ as $X = (x_1', x_2', ... x_N')'$. That is, you build a regressor matrix $X$ where each row corresponds to an individual $i$ whose measurements of $x_1$, $x_2$ you have taken. The first column consists of $1$s and takes into account that you have an intercept. One can now rewrite for $Y = (Y_1, Y_2, ... Y_N)'$, $e = (e_1, ... e_N)'$ and $\beta = (a,c,b)$ your regression as $Y = X\beta + e$. (This single equation represents $N$ equations, one for each individual.) Then, as before, $(X'X)^{-1}X'Y = \hat{\beta} = (\hat{a}, \hat{c}, \hat{b})'$ is the OLS estimator.

Finding the Coefficients of regressors

1 Answers1