1

Context

This question emerged from trying to solve problem 5.1. of Wooldridge, Econometric Analysis of Cross Section and Panel Data.

The problem asks to show the equivalence of the estimators obtained via a 2-stage least square procedure and the estimators obtained via an OLS regression to which the residuals of the auxiliary regression from the 2SLS are added as regressors.

In an attempt to solve the problem I stumbled upon the question below. I found another way to solve the problem which does not require an answer to the following question, but I am still interested in the question itself. Here it goes.

Question

Suppose the column vectors of the matrix $X$ form the basis of some space. Decompose $X$ into two sub matrices

$$ X = ( X_1~X_2) .$$

Assume that the the vector $y$ is orthogonal to every column vector of $X_1$. Suppose that we want to project $y$ onto the space orthogonal to the column space of $X$ (i.e. the null-space of $X'$). This yields the vector

$$ p =( I - X(X'X)^{-1} X') y.$$

I have a strong geometrical intuition that because $y$ is already orthogonal to the vectors in $X_1$, this is the same as projecting $p$ onto the space orthogonal to the column space of $X_2$ ( i.e. the null-space of $X_2'$). Formally this would mean

$$ ( I - X(X'X)^{-1} X') y = ( I - X_2(X_2'X_2)^{-1} X_2') y. $$

I found some examples where it works, but I could not find an algebraic proof or a counter-example (I tried using decompositions of $(X'X)^{-1}$ into terms depending only on $X_1$ and $X_2$ but to no avail). Is this true? Any idea how to prove or disprove this algebraically?

Martin Van der Linden
  • 739
  • 1
  • 6
  • 13
  • 2
    The analysis you are looking for has been presented in my answer at http://stats.stackexchange.com/a/113207/919. It provides a geometric demonstration (which is readily converted into algebra if you really must). – whuber Sep 19 '14 at 16:11
  • Thanks @whuber. My intuition for the result was precisely geometrical and followed the kind of arguments you provided in your other answer you link to. My problem is precisely about the conversion of the argument to algebra. I believe you when you say that it can readily be converted, but I have a hard time figure out how. I edited the question to make this clearer. – Martin Van der Linden Sep 19 '14 at 16:40
  • 2
    The key idea present in the geometric analysis is that $X_1$ needs to be taken out of $X_2$ (via comparable regression formulas). By doing so, $X_2$ will be changed to a matrix whose columns are orthogonal to those of $X_1$ *without changing the projection at all*. Thus, with no loss of generality, you may assume $X_1$ and $X_2$ are orthogonal. This implies $X^\prime X$ has a block-matrix form with zero-matrices in the off-diagonal positions. The left hand side of your intended identity immediately reduces to the right hand side when $y$ is orthogonal to $X_1$. – whuber Sep 19 '14 at 18:12
  • Thanks @whuber. I'll try to work it out based on your indications. – Martin Van der Linden Sep 19 '14 at 18:18

0 Answers0