Linear model trace of the hat matrix in R

Question

This question is about the difference between the sum of lm.influence(model)$hat and the trace of the Hat-Matrix $H := X (X' X)^{-1} X'$ calculated "by hand".

Let's take for example the cars data-set and the linear-model $dist = \beta \cdot speed + \varepsilon$

data(cars)
mod <- lm(dist ~ speed, cars)

Using lm.influence to calculate the trace I get 2. lm.influence is supposed to return the diagonal elements of $H$ (see ?lm.influence).

sum(lm.influence(mod)$hat)
[1] 2

Calculating the trace of $H$ "by hand" I get 1:

X <- cars$speed
H <- X %*% solve(t(X) %*% X) %*% t(X)
sum(diag(H))
[1] 1

From my understanding it should be 1 as the Rank of a 1-column-vector $\equiv$ 1.

For linear models, the trace of the hat matrix is equal to the rank of X, which is the number of independent parameters of the linear model. Source: http://en.wikipedia.org/wiki/Hat_matrix

Answer 1

In R, the model lm(dist ~ speed, cars) includes an intercept term automatically. So you have actually fitted $dist = \beta_0 + \beta_1 speed + \epsilon$.

You rarely want to drop the intercept term but if you did, you can do lm(dist ~ 0 + speed, cars) or even lm(dist ~ speed - 1, cars). See this question on Stack Overflow for more details.

Without the intercept term, we get the result that matches what you got "by hand":

> data(cars)
> mod <- lm(dist ~ 0 + speed, cars)
> sum(lm.influence(mod)$hat)
[1] 1

You can see this geometrically by considering the hat matrix as an orthogonal projection onto the column space of $X$. With the intercept term in place, the column space of $X$ is spanned by $\mathbf{1}_n$ and the vector of observations for your explanatory variable, so forms a two-dimensional flat. An orthogonal projection onto a two-dimensional space has rank 2 - set up your basis vectors so that two lie in the flat and the others are orthogonal to it, and its matrix representation simplifies to $H = \text{diag}(1,1,0,0,...,0). $ This clearly has rank 2 and trace 2; note that both of these are preserved under change of basis, so apply to your original hat matrix too.

Dropping the intercept term is like dropping the $\mathbf{1}_n$ from the spanning set, so now your column space of $X$ is one-dimensional. By a similar argument the hat matrix will have rank 1 and trace 1.