2

I'm looking the snippet below from ESL. I'm a hard time deriving the variance term (last term in Eq. 7.12).

I started with \begin{align} \frac{1}{N}\sum_i ||h(x_i)||^2\sigma^2 \\ = \frac{1}{N}\sigma^2\sum_i ||h(x_i)||^2 \\ = \frac{1}{N} \sigma^2 \sum_i x_i^T(X^TX)^{-1}x_i\\ \end{align}

If I compare my last expression and their's it would seem the summation is equal to $p$. Assuming my above steps are correct, I can't see how this simplifies to $p$.

enter image description here

David
  • 563
  • 2
  • 6

1 Answers1

2

The key step is to note that the sum is the trace of the hat matrix, i.e. $$ \sum_i x_i^T(X^TX)^{-1}x_i = \text{tr}\left(X(X^TX)^{-1}X^T\right) \\ = \text{tr}((X^TX)^{-1}X^TX) = \text{tr}(I_p) = p. $$

jld
  • 18,405
  • 2
  • 52
  • 65