1

In a univariate regression, $Y=a+bX+e$, the solution for slope b is given by $COV(X,Y)/VAR(X)$.

Is there a similar expression for a bivariate regression $Y=a+bX+cZ+e$. What is the closed form solution for b and c in this case?

user95626
  • 13
  • 2
  • Terminology: *bivariate* regression would normally refer to the case where there are a pair of response variates ($Y_1$ and $Y_2$, say), rather than two predictors; note that when there are multiple predictors ("x"-variables) it is called [multiple regression](http://mathworld.wolfram.com/MultipleRegression.html), not [multivariate regression](https://en.wikipedia.org/wiki/Multivariate_statistics#Types_of_analysis) (see item 2 there). – Glen_b Nov 19 '15 at 22:43

2 Answers2

6

Yes, it is called ordinary least squares (OLS) and it is covered in most handbooks:

$$ \hat{\boldsymbol\beta} = (\mathbf{X}^{\rm T}\mathbf{X})^{-1} \mathbf{X}^{\rm T}\mathbf{y} $$

where $\hat{\boldsymbol\beta}$ and $\mathbf{X}$ are matrices and $\mathbf{y}$ is a vector. See also this thread.

Community
  • 1
Tim
  • 108,699
  • 20
  • 212
  • 390
  • Thanks, this is pointing in the right direction. What would be the result for a bivariate regression (with 1 intercept and 2 regressors) without using matrix notation? I am looking for an expression just in terms of covariances and variances. – user95626 Nov 19 '15 at 14:39
  • You have covariance of three variables (X,Y,Z) and their variances, you are interested in their multivariate relations and so things are getting messy, so we are using mathematical tools like matrix algebra in here. – Tim Nov 19 '15 at 14:45
  • 3
    It's actually much simpler to use the matrix algebra; indeed, once you follow it, it's also the easier way to look at simple linear regression. – Glen_b Nov 19 '15 at 22:45
2

Given $$y=\beta_1+\beta_2 x_2 + \beta_3 x_3 +\varepsilon $$ with the usual assumptions, OLS would correspond to solving these normal equations

$$ n \cdot b_1 + b_2 \sum x_2 + b_3 \sum x_3 = \sum y \\ b_1 \sum x_2 + b_2 \sum x_2^2 + b_3 \sum x_2 \cdot x_3= \sum x_2 \cdot y \\ b_1 \sum x_3 + b_2 \sum x_2 \cdot x_3 + b_3 \sum x_3^2= \sum x_3 \cdot y, $$

where $n$ is the sample size. I don't think there's a neat interpretation like there is with the 2D case.

dimitriy
  • 31,081
  • 5
  • 63
  • 138