Gauss-Markov theorem: BLUE and OLS

Question

I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem.

We assume a linear model, in matrix form, is given by: $$ y = X\beta +\eta $$ and we're looking for the BLUE, $ \widehat\beta $.

In accordance with this, I would label $\eta = y - X\beta$ the "residual" and $\varepsilon = \widehat\beta - \beta$ the "error". (I.e the opposite of the usage on the Gauss-Markov page).

The OLS (ordinary least-squares) estimator may be derived as the argmin of $||\text{residual}||_2^2 = ||\eta||_2^2$.

Now, let $\mathbb{E}$ denote the expectation operator. To my understanding, what the Gauss-Markov theorem tells us is that, if $\mathbb{E}(\eta) = 0$ and $\text{Var}(\eta) = \sigma^2 I $, then the argmin, over all linear, unbiased estimators, of $\mathbb{E}(||\text{error}||_2^2) = \mathbb{E} (||\varepsilon||_2^2)$ is given by the same expression as the OLS estimator.

I.e. $$ \text{argmin}_{\text{} \widehat\beta(y)} \, ||\eta||_2^2 \;=\; (X'X)^{-1}X'y \;=\; \text{argmin}_{\text{linear, unbiased } \widehat\beta(y)} \, \mathbb{E}(||\varepsilon||_2^2)$$

Is my understanding correct? And if so, would you say that it deserves more prominent emphasis in the article?

Answer 1

I am not sure if I understood you question correctly, but if you are looking to prove that the OLS for $\hat{\beta}$ is BLUE (best linear unbiased estimator) you have to prove the following two things: First that $\hat{\beta}$ is unbiased and second that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators.

Proof that OLS estimator is unbiased can be found here http://economictheoryblog.com/2015/02/19/ols_estimator/

and proof that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators can be found here http://economictheoryblog.com/2015/02/26/markov_theorem/

Answer 2

It seems my hunch was correct indeed, as confirmed, e.g. on page 375 of the book Introductory Econometrics. Relevant excerpt: