Frisch-Waugh-Lovell theorem: How do we know this matrix is invertible?

Question

The frisch-waugh-lovell theorem uses the following line in the proof:

$\hat\beta_2 = (X_2' M_{1}X_2)^{-1}X_2'M_{1}y$

How do we know

$(X_2' M_{1}X_2)^{-1}$

is invertible?

Do you know anything specific about $M_1$? Because that isn't invertible in general. — Sycorax, Mar 24 '17 at 15:36
You don't know it's invertible--but you don't invert it in the first place. That's just a notation for solving a set of linear equations. Equivalently, think of it as a generalized inverse. — whuber, Mar 24 '17 at 15:43
@whuber I would really appreciate it if you could have a look at my attempt at a proof below. Thank you. — Alecos Papadopoulos, Mar 24 '17 at 17:15

score 1 · Answer 1 · answered Mar 24 '17 at 17:14

I think that we can show that the matrix is invertible if the full regressor matrix has full column rank, but please check my proof.

We are looking at a regression with $k_1+k_2$ regressors (counting a possible constant term) having a sample of size $n\geq k_1+k_2$. We asuume that the full regressor matrix $X = [X_1:X_2]$ has full-column rank equal to $k_1+k_2$. This means that $X_1$ has column rank $k_1$ and $X_2$ has column rank $k_2$.

Let $M_1 = I_{n} - P_1,\,\, P_1 = X_1(X_1'X_1)^{-1}X_1'$, the residual maker and orthogonal projection matrices respectively, symmetric and idempotent both. We want to show that the $n \times k_2$ matrix $M_1X_2$ has full column rank $k_2$, so that its Grammian matrix $[(M_1X_2)'(M_1X_2)] = X_2'M_1X_2$ is invertible.

Assume that $M_1X_2 = W$ has not full column rank. This means that at least two columns of $W=M_1X_2$ are linearly dependent, say columns $w_1 = x_{21}- P_1x_{21}$ and $w_2= x_{22} - P_1x_{22}$.
If they are linearly dependent it means that for some real scalar $a$ we have

$$w_1 = aw_2 \implies x_{21}- P_1x_{21}= ax_{22} - P_1ax_{22}$$

$$\implies x_{21}- ax_{22} = P_1(x_{21} - ax_{22})$$

So if $w_1$ and $w_2$ are linearly dependent, we have obtained that the Projection matrix $P_1$ must act as the identity matrix on the column vector $x_{21}- ax_{22}$. But in order to do so, this column vector must be inlcuded in the regressors used to form $P_1$.

But this implies that in the full regressor matrix $X$, there exists a column which is a linear combination of two other columns, and so $X$ has not full column rank, and so $X'X$ is not invertible to begin with.

So what I think I have proved is the following:

If the full regressor matrix $X$ has full column rank, then the matrix $M_1X_2$ as defined above has also full column rank.

+1 I believe you, because this theorem is an immediate consequence of an easily proven generalization to arbitrary partitions of the set of explanatory variables. (I describe this generalization geometrically at http://stats.stackexchange.com/a/113207/919 and algebraically, in terms that look similar to yours, at http://stats.stackexchange.com/a/46508/919.) — whuber, Mar 24 '17 at 18:22

Frisch-Waugh-Lovell theorem: How do we know this matrix is invertible?

1 Answers1