Kruskal–Wallis one-way analysis of variance is related to what kind of regression?

Question

One-way anova is similar to regular linear regression because both use the F-test which involves sums of squares among other reasons.

Is Kruskal–Wallis one-way analysis of variance similar to some kind of non-parametric regression? If so, what?

Answer 1

I haven't seen anyone do this before, but I think we can say some useful things, while not getting a complete solution (indeed a complete regression procedure that also yields KW when you have a single factor predictor may not be possible).

You can estimate relative shifts in a Kruskal Wallis framework easily enough; simply choose the (relative) location shifts that minimize the statistic. These are not necessarily unique; sometimes there's an interval for one of the shifts or even a region of several shifts which all give the same minimum value of the statistic. However, this is usually fairly readily resolved (e.g. for an interval, by taking the center point). It's also possible to obtain a joint confidence region for the shift parameters (see the bottom of the answer here Difference Between ANOVA and Kruskal-Wallis test).

However, this doesn't determine the intercept (the Kruskal-Wallis statistic deals with relative arrangements, not absolute positions). Choosing a baseline group and then using some suitable measure of location for it is somewhat unsatisfying in that a different choice of baseline could result in a different set of fitted locations.

We could take some location estimate based off all the shifted-to-equality groups, though there's some aribtrariness in the choice of the estimator.

We might even use the Kruskal-Wallis statistic itself to do so. This relies on an idea that turns bivariate statistics into univariate ones -- e.g. a way of testing (and also thereby estimating) location of a single group via a statistic for two groups. Which is to say, for some hypothesized location θ₀, construct a new set of data Yᵢ=2θ₀-Xᵢ and perform a two-sample test of equality of location on X,Y*. Location would be estimated by identifying the θ₀ that corresponded to the smallest possible location difference.

We could potentially use the same idea with the Kruskal Wallis after equalizing the locations by inverting all groups through θ₀ (at a given θ₀, for each X create a Y in the above fashion), and then find the θ₀ that minimizes this new Kruskal Wallis statistic. Since this is symmetric in the labelling of the groups it should not be affected by which group was chosen as the baseline. [However, I expect this would correspond to a much simpler calculation.]

However we handle the intercept, this does seem to give us a "regression" (in the sense of a set of parameter estimates) that corresponds to a Kruskal-Wallis, though we don't automatically have all the same information we'd get with a regression, and we'd like it to be invariant (or equivariant, as appropriate) to the choice of the baseline group. This won't necessarily be possible for everything we'd like to get from a regression-type calculation.

Further, this doesn't readily extend to the general case of regression -- as currently presented, it is limited to "regression" on a single factor.

* I wish I could locate the paper I first saw this in, I believe some time in the 80s (though it might just have been published a little earlier). I thought it was in The American Statistician or possibly JASA, but I wasn't able to locate it in either yesterday while writing this answer.