5

When I have a design matrix $$X = \begin{bmatrix} 1 & x_{11} & \ldots & x_{1k}\\ 1 & x_{21} & \ldots & x_{2k}\\ \vdots & \vdots & \ddots & \vdots\\ 1 & x_{n1} &\ldots & x_{nk} \end{bmatrix} $$ how can I show that subtracting the mean from columns $2$ to $k$ does not change the hat matrix given by $$H = X (X^T X)^{-1} X^T \,\,?$$

Chill2Macht
  • 5,639
  • 4
  • 25
  • 51
ira
  • 399
  • 2
  • 14

1 Answers1

2

If you demean the data matrix $X = [1_n,X_2]$, you get the new data matrix $$ \tilde{X} = [1_n, \tilde{X}_2], $$ where $\tilde{X}_2 = [I_n - n^{-1} 1_n 1_n^T]X_2 = DX_2$.

This is all just inverses of block matrices. Hopefully I didn't make a mistake. There's probably a cleaner way to do this.

The (possibly new) hat matrix is \begin{align*} H_2 &= \tilde{X} (\tilde{X}^T \tilde{X})^{-1}\tilde{X}^T \\ &= [1_n, \tilde{X}_2] \left[\begin{array}{cc} n & 1_n^T \tilde{X}_2 \\ \tilde{X}_2^T 1_n & \tilde{X}_2^T\tilde{X}_2 \end{array}\right]^{-1} \left[ \begin{array}{c} 1_n^T \\ \tilde{X}_2^T \end{array} \right] \\ &= [1_n, DX_2] \left[\begin{array}{cc} n & 1_n^T DX_2 \\ X_2^TD^T 1_n & X_2^TD^TDX_2 \end{array}\right]^{-1} \left[ \begin{array}{c} 1_n^T \\ X_2^TD^T \end{array} \right] \\ &= [1_n, DX_2] \left[\begin{array}{cc} n & 0 \\ 0 & X_2^TDX_2 \end{array}\right]^{-1} \left[ \begin{array}{c} 1_n^T \\ X_2^TD^T \end{array} \right] \\ &= [1_n, DX_2] \left[\begin{array}{cc} n^{-1} & 0 \\ 0 & (X_2^TDX_2)^{-1} \end{array}\right] \left[ \begin{array}{c} 1_n^T \\ X_2^TD^T \end{array} \right] \\ &= [n^{-1} 1_n, DX_2(X_2^TDX_2)^{-1}] \left[ \begin{array}{c} 1_n^T \\ X_2^TD^T \end{array} \right] \\ &= n^{-1}1_n 1_n^T + DX_2(X_2^TDX_2)^{-1} X_2^TD^T \\ &= n^{-1}1_n1_n^T + n^{-2}1_n1_n^TX_2[X_2^TDX_2 ]^{-1}X_2^T1_n1_n^T - X_2X_2^TX_2 X_2^T 1_n[X_2^TDX_2]^{-1}1_n^T -n^{-1}1_n1_n^T X_2 [X_2^TDX_2 ]^{-1}X_2^T +X_2[X_2^TDX_2]^{-1}X_2^T \\ &= [n^{-1}1_n + n^{-2}1_n1_n^TX_2[X_2^TDX_2 ]^{-1}X_2^T1_n - X_2X_2^TX_2 X_2^T 1_n[X_2^TDX_2]^{-1}, -n^{-1}1_n1_n^T X_2 [X_2^TDX_2 ]^{-1} +X_2[X_2^TDX_2]^{-1}] \left[ \begin{array}{c} 1_n^T \\ X_2^T \end{array} \right] \\ &= [1_n, X_2] \\ &\left[\begin{array}{cc} n^{-1} + n^{-2}1_n^TX_2[X_2^TDX_2 ]^{-1}X_2^T1_n & -n^{-1}1_n^T X_2 [X_2^TDX_2 ]^{-1} \\ - X_2^TX_2 X_2^T 1_n[X_2^TDX_2]^{-1} & [X_2^TDX_2]^{-1} \end{array}\right] \left[ \begin{array}{c} 1_n^T \\ X_2^T \end{array} \right] \\ &= [1_n, X_2] \\ &\left[\begin{array}{cc} n^{-1} + n^{-2}1_n^TX_2[X_2^TX_2 - n^{-1}X_2^T1_n1_n^T X_2]^{-1}X_2^T1_n & -n^{-1}1_n^T X_2 [X_2^TX_2 - n^{-1}X_2^T1_n1_n^T X_2]^{-1} \\ - X_2^TX_2 X_2^T 1_n[X_2^TX_2 - n^{-1}X_2^T1_n1_n^T X_2]^{-1} & [X_2^TX_2 - n^{-1}X_2^T1_n1_n^T X_2]^{-1} \end{array}\right] \left[ \begin{array}{c} 1_n^T \\ X_2^T \end{array} \right] \\ &= [1_n, X_2] \left[\begin{array}{cc} n & 1_n^T X_2 \\ X_2^T 1_n & X_2^TX_2 \end{array}\right]^{-1} \left[ \begin{array}{c} 1_n^T \\ X_2^T \end{array} \right] \\ &= X(X^TX)^{-1}X^T \end{align*}

Taylor
  • 18,278
  • 2
  • 31
  • 66