Learning hat matrix

Question

I'm stuck learning the hat matrix and wondered if someone could help with a question. If I have the model $$Y_i =\beta_0+\beta_1X_i+\epsilon_i,i = 1,2,3 \dots n,$$ how can I calculate the hat matrix as

$$H = X(X^\prime X)^{-1}X^\prime$$

And what would the $h_{ij}$ element be?

https://en.wikipedia.org/wiki/Hat_matrix This looks like it will help. — Eric Peterson, Oct 28 '15 at 20:25
(1) That minus sign does not belong in the expression for $H$. I guess you intend it to mean an inverse or a pseudo-inverse of the quantity within parentheses. (2) Are you perhaps asking what a matrix is and how to multiply matrices? — whuber, Oct 28 '15 at 20:39
Sorry I have corrected it now. Yes please whatever is relevant to solve the question — user3443632, Oct 28 '15 at 20:42
Assuming that your linear model is `mod` in R you would write something like: `X = model.matrix(mod); (H = (X %*% solve(crossprod(X)) %*% t(X)))`. See this thread [here](http://stats.stackexchange.com/questions/125886/linear-model-trace-of-the-hat-matrix-in-r) for more details. — usεr11852, Oct 28 '15 at 21:42
It's unclear what you are asking. You can calculate the hat matrix using the formula that you wrote down. Is there something particular about that formula that you do not understand? — Matthew Drury, Oct 29 '15 at 00:41
The notation $h_{ij}$ denotes the entry in the matrix $H$ in the $i$th column and $j$th row. The matrix $H$ can be calculated as in your formula. Are you looking for a formula that gives $h_{ij}$ directly? Any such formula would just be expanding the matrix formula you gave, and would not be particularly useful or insightful. — Matthew Drury, Oct 29 '15 at 00:45
Using the definition of matrix multiplication, $h_{ij} = \sum_{km} x_{ik} z_{km} x_{jm}$, where $Z = (X' X)^{-1}$. There is no legitimate closed form for $z_{km}$ without giving in to absurdity. — Matthew Drury, Oct 29 '15 at 00:50
I have merged the two copies of your question so that all comments appear in one place. If you would like to make changes to this question, then please edit it rather than creating a new one. — whuber, Oct 29 '15 at 03:28

score 2 · Answer 1 · answered Mar 08 '19 at 21:48

We can write the model in matrix notation as

$$ Y = \beta X + \epsilon $$

The OLS solution for the vector of regression coefficients $\beta$ is:

$$ \hat{\beta} = (X'X)^{-1} X'Y $$

The hat matrix is the projection matrix that maps the response vector $Y$ to the vector of fitted values $\hat{Y}$ (hence the name "hat" matrix). That is:

$$ \hat{Y} = HY$$

Now, since
$$ \hat{Y} = X \hat{\beta} = X(X'X)^{-1} X'Y $$

it immediately follows that

$$ H = X(X'X)^{-1} X'$$

as required.

Learning hat matrix

1 Answers1