6

What exactly does Cook's distance measure? And how is this different from what hat values measure?

I know hat values measure how distant a point it form its corresponding fitted point. I also know Cook's distance measures the influence of a point (whether it changes the fitted line) but what exactly does it measure?

I other words, what exactly is the difference between hat values and Cook's distance?

Maxim
  • 3,164
  • 1
  • 17
  • 25
Sara
  • 109
  • 1
  • 1
  • 3
  • 2
    Does the Wikipedia entry https://en.wikipedia.org/wiki/Cook%27s_distance help you? perhaps come back and edit your question with a more specific query as it is a bit hard to see what level of explanation you are looking for. – mdewey Dec 15 '17 at 15:15

2 Answers2

5

The cook's distance is given by the formula: $D_{i} = \frac{\sum_{j = 1}^{n} (\hat Y_j - \hat Y_{j(i)})^2}{pMSE}$

Where:

  • $\hat Y_j$ is the fitted value for the j observation;
  • $ \hat Y_{j(i)}$ is the fitted value for the j observation without including the i-th observation in the data that will generate the model;
  • p is the number of parameters in the model;
  • MSE ie the mean squared error of the model.

This means that the cook's distance measures the influence of each observation in the model,or "what would happen if each observation wasn't in the model", and it's important because it's one way of detecting outliers that affects specially the regression line. When we don't look for and treat potential outliers in our data, it is possible that the adjusted coefficients for the model might not be the most representative, or appropriate, leading to incorrect inference.

The hat values are the fitted values, or the predictions made by the model for each observation. It is quite different from the Cook's distance.

Bruna w
  • 513
  • 2
  • 9
4

Cooks distance shows how much the whole regression model would change if $(x_i, y_i)$ is removed. I am not quite clear what you mean by "hat value". Do you mean $e_i = y_i - \hat{y}_i$, or $h_{ii}$ in the hat matrix $H$ (i.e. leverage)? Either way they are different from cooks distance.

Note that cooks distance takes the form $$D_i = \frac{e_{i}^{2}}{s^{2} p}\left[\frac{h_{ii}}{(1-h_{ii})^2}\right]$$, so it's related to both residual $e_i$ and leverage $h_{ii}$. Large $D_i$ could be due to large $e_i$ or $h_{ii}$, or both.

Possible reasons for

large residual $e_i$: $y_i$ far from fitted value (possibly outlier)

large leverage $h_{ii}$: $x_i$ far from other $x_{j}$'s (influential point due to the value of $x$)

Lii
  • 1,026
  • 1
  • 8
  • 11