5

This is a bit of a terminology question, but what is the difference between a robustness check and a sensitivity analysis? For example, if performing analysis to see how sensitive (or robust) a study's conclusions are to additional variables.

Are robustness checks a type of sensitivity analysis or vice versa?

asdf
  • 59
  • 1
  • 3
  • 4
    A bit more context might help. – Scortchi - Reinstate Monica Mar 05 '14 at 13:13
  • 1
    As far as I know, sensitivity analysis has to do with the values of variables that are in the model: how much changes in them affect the output of the model. If you're talking about how sensitive the results are to what variables are included, you are really doing more of a model comparison. – Wayne Mar 05 '14 at 19:18

2 Answers2

1

I don't know of an 'official' answer to this based on universally accepted definitions of those terms. However, it seems to me that they represent a fundamentally similar idea, but are used somewhat differently.

  • "Robustness check" is often used when running a different model / test that does not require a certain assumption. For example, consider a situation where you are comparing two groups where there may be heteroscedasticity. You could run a standard $t$-test and the Welch $t$-test. If you get the same result both ways, you could say your result is robust to violations of that assumption—it just isn't something you need to be overly worried about.

  • "Sensitivity analysis" is often used in the context of missing data. Many convenient methods are valid if data are missing at random (MAR), but you can never really be certain that your data are MAR. A way to explore this is to input different values that might be problematic / related to the missingness and refit the model. Again, it is comforting if you get the same result both ways.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
0

Here is the answer your are looking for: 1. A robustness check means that your results are not highly determined by changes to your dataset (i.e. you could use a similar data set, or group your data slightly differently, and still get similar results). 2. Sensitivity analysis means that your results are not highly determined by your model specification (i.e. you could add an additional control variable, or a slightly different functional form, and still get similar results).

Thus, (1) is how stable your results are to inputs and (2) is how reactive your results are to design.