Difference between DFBETA with DFFITS / Cook’s distance

Question

A description of DFBETA is on the Wikipedia page: https://en.wikipedia.org/wiki/Influential_observation along with DFFITS and Cook’s distance.

Cook’s distance and DFFITS are conceptually identical and there is a formula to convert one value to the other. Thus DFBETA could be viewed as more different to DFFITS and Cook’s distance.

I am interested in benefits of what DFBETA gives you over DFFITS / Cook’s distance, and vice versa, or guidance of when to prefer one or the other—although of course you can use DFBETA and DFFITS / Cook’s distance as multiplicity issues do not apply here as far as I can tell.

Because DFBETA shows how the model coefficients change it gives an idea or model stability: small changes in coefficients mean a more stable model, suggesting a lower variance in out of sample estimates. But this benefit of DFBETA may also be present with Cook’s distance:

$D_i$ can be interpreted as the distance one's estimates move within the confidence ellipsoid that represents a region of plausible values for the parameters. This is shown by an alternative but equivalent representation of Cook's distance in terms of changes to the estimates of the regression parameters between the cases, where the particular observation is either included or excluded from the regression analysis.

[https://en.wikipedia.org/wiki/Cook%27s_distance]

Answer 1

This question was answered in a question concerning Cook's distance How to read Cook's distance plots? but I will provide a summary for expository purposes.

First, the terms DFFIT and DFBETA mean "difference in fit" and "difference in beta". These are descriptive terms, being how the non-inclusion of an individual observation changes respectively its fitted value and the beta coefficients. DFFIT is the difference in fit of removal of an individual observation whereas Cook's D is the average change of a fit of an individual observation. From https://stats.idre.ucla.edu/stata/webbooks/reg/chapter2/stata-webbooksregressionwith-statachapter-2-regression-diagnostics/ Cook’s D and DFITS [note that Stata uses a single "F" in the command for DFFIT] "measures both combine information on the residual and leverage. Cook’s D and DFITS are very similar except that they scale differently but they give us similar answers.". Thus in practice Cook's D and DFITS are similar.

As @whuber says in a comment "I would add that influential cases are not usually a problem when their removal from the dataset would leave the parameter estimates essentially unchanged: the ones we worry about are those whose presence really does change the results.". This suggests that when I asked for "guidance of when to prefer one or the other" in the question this is a false dichotomy, as they do different things. A typical workflow may be something like first to run DFFIT or Cook's distance, and use this to investigate influential data points, which can be useful information to understand your data set better. If there are influential points, you can then run DFBETA and investigate if these change the betas. Alternatively if all you care about is "stability" of regressor coefficients you might just run DFBETA.

@gung-ReinstateMonica says in the answer "Cook's distance can be contrasted with dfbeta. Cook's distance refers to how far, on average, predicted y-values will move if the observation in question is dropped from the data set. dfbeta refers to how much a parameter estimate changes if the observation in question is dropped from the data set.". Also "Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling.". I interpret "explanatory modeling" to be part of statistical inference.

The final paragraph of the answer is useful perspective and is copied below in full. "There is one other point worth making here. In observational research, it is often difficult to sample uniformly across the predictor space, and you might have just a few points in a given area. Such points can diverge from the rest. Having a few, distinct cases can be discomfiting, but merit considerable thought before being relegated outliers. There may legitimately be an interaction amongst the predictors, or the system may shift to behave differently when predictor values become extreme. In addition, they may be able to help you untangle the effects of colinear predictors. Influential points could be a blessing in disguise.".