1

Say that I have two distributions. Both are very skewed distributions that don't seem to fit any distribution I know well.

Should I turn to a non-parametric (distributionless) test or transform the datasets to a distribution that I know (for example, box-cox on both distirbutions then T-test)?

Note that we are testing equality of the distributions.

  • 1
    What do you want to test? – Christoph Hanck Mar 23 '16 at 19:11
  • Whether the means are different. But it would be nice for a general answer to improve any misunderstandings I may have –  Mar 23 '16 at 19:18
  • 2
    Well, the answer depends on the aim. If you wanted to test equality of distribution, a nonparametric approach seems appropriate, whereas testing equality of means does not require symmetric distributions. – Christoph Hanck Mar 23 '16 at 19:23
  • Interesting. I didn't know that. Let's go with testing equality of the distributions. –  Mar 23 '16 at 19:24
  • I think you should see the popular post on how to [Identify probability distributions](http://stats.stackexchange.com/q/10517/109497). – Inon Mar 23 '16 at 19:47

1 Answers1

2

You might test equality of distribution against an alternative of a difference in means via a permutation test (if you're prepared to assume that under the null of equal means the distributions would be the same).

You could test equality of distributions against one variable being stochastically larger via a Wilcoxon-Mann-Whitney.

You could test equality of distribution vs a more general alternative via a two-sample goodness of fit test (e.g. two sample Kolmogorov-Smirnov).

There are many other possible choices.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thank you for the response! All those reasons make sense. `There are many other possible choices.` It would be awesome if you could briefly list those so I can read into them. –  Mar 24 '16 at 01:47
  • Why not use a transformation and transform it to a known distribution? For example, if both these distributions are skewed: why not boxcox them both - then use Student's t-test on these transformed? Are there any trade-offs to transforming vs. non-parametric approaches? –  Mar 24 '16 at 01:49
  • 1
    1. If you don't know the distribution you start with, how exactly are you going to end up with a random sample from a normal distribution? (I remind you of your own question about difficulties of transforming to symmetry, let alone normality) $\:$ 2. Even if you did, after transforming you wouldn't be comparing means (so if you wanted to say something about how much the mean changed the usual confidence interval on that transformed scale doesn't work.$\:$ 3. If you *do* know the distribution so you could transform, why not just use an efficient test for whatever the parameters of that one are? – Glen_b Mar 24 '16 at 03:29
  • But the transformation idea makes sense in insofar as comparison is better done on transformed scales. If two groups differ multiplicatively, then the relevant comparison is on log scale: instead of comparing two different means and SDs, the SDs on log scale may be similar and the focus is then on means. A framework of generalised linear models is convenient for this: you can cycle over different links and/or families and find (sometimes) that you get the same answer even entertaining different models for the data. Finding that you don't get the same answer would be informative too. – Nick Cox Mar 24 '16 at 13:10
  • @Nick Yes, I definitely agree that there are reasons to transform where that makes sense for the thing you want to compare, that's a natural reason to take logs, for example, though it won't necessarily make anything normal. – Glen_b Mar 25 '16 at 09:04
  • 1
    Quite. The point has been made many times, e.g. in http://stats.stackexchange.com/questions/203258/when-does-taking-the-log-transformation-of-a-univariate-not-remove-skew – Nick Cox Mar 25 '16 at 09:07