Is there a robust test for stochastic dominance between two random variables?

Question

I am trying to compare the errors from two statistical models in order to give evidence to one being "better" in terms of lower prediction error than the other.

To formalize this, I thought that a test of stochastic dominance between two collections of random variables (the OOS errors) would be a good idea. Ideally the null hypothesis would be :

$$\mathbb{H}_0: F(x) \ge G(x) \forall x \in \mathbb{R} $$

I have found resources pointing me to the Kruskal-Wallis test, but unfortunately cannot seem to find a paper explicitly stating and proving one of these (or similar) null hypotheses. Many sources I check simply state that the null is that the medians of the two distributions differ, but this is not what I want to check. Any help is appreciated.

Answer 1

I don't see that you need a special proof for this. It is foundational that these nonparametric tests are testing for stochastic dominance. If you need a reference to cite, I would just use a basic nonparametric statistics textbook. I tend to use Hollander & Wolfe (2013) for that sort of thing.

Regarding your situation, here are a couple additional points:

1. The Kruskal-Wallis test generalizes the Mann-Whitney U-test to more than two groups, but you only seem to have two, so it's not really necessary.
2. The Krukal-Wallis, and the Mann-Whitney, are for independent groups, but you presumably have paired data. That is, you have the prediction error for a given observation from each of two models. Those two errors are meaningfully paired. You need to take that into account, so something like the the Wilcoxon signed rank test is presumably appropriate.
3. These tests are for stochastic dominance, but that isn't quite the same as being tests of medians. It is common to call them tests of medians, but that is only true under very restrictive circumstances (cf, What exactly does a non-parametric test accomplish & What do you do with the results?).
4. Lastly, it isn't clear what you mean by "robust test for stochastic dominance" in the title. Nonparametric tests are already robust in the sense typically meant in statistics.