1

I have observations from an experiment, and each of the observations belongs to one of 2 groups (patients, controls).

I would like to test for group differences in the data using a test. I would normally use a 2-sample t-test. However, in this case the data does not follow a Normal distribution. Here is a histogram of the data for one of the two groups:

Histogram of data

There are too many outliers, and also there is a very high peak at the mean.

What would be the best approach in this case? Is there any transformation I could apply to the data before running the test (and that would make it more normal), or should I not even try that and instead use a test that does not assume normality of the data? What are the pros and cons of data transformation in this case?

John Silver
  • 65
  • 7
  • If you can post the data, then people can find out whether their prejudices, preferences and predictions are matched by experience. – Nick Cox Mar 29 '19 at 16:05
  • What aspect of the distribution are you interested in comparing? (alternatively, what's the working hypothesis?) – Glen_b Mar 29 '19 at 16:36
  • I am interested in finding out whether there are group differences in the mean, but as I need to covariate for demographics data (e.g., age, and site effects) I am using a GLM for that. I would have to talk to the legal department before posting the raw data, I guess. I guess I could post the measurements shown in the histogram without any problem, but I am not so sure about the demographics data. – John Silver Apr 01 '19 at 09:24
  • I've never seen "covariate" as a verb before. Sure, if you wish to go further and model using other covariates, then that raises different question, principally whether the relationship is linear or something else. Nothing about that in the question! – Nick Cox Apr 01 '19 at 10:37
  • Yes, I left that out of the question because I felt it would complicate things and I was mainly interested in the question as it stands now: whether applying a data transformation before a test can make sense at all, and what are the pros and cons of doing so. On using *covariate* as a verb: sorry, English is not my primary language. I meant modeling using covariates. – John Silver Apr 01 '19 at 12:45
  • The language is changing, and more people use English than learn it as a small child. I see "I have a data" and feel pain, but perhaps the language is moving that way. "Data is" no longer causes pain, despite what my Latin teachers said 50+ years ago. – Nick Cox Apr 01 '19 at 14:54

2 Answers2

5

I disagree somewhat with the first answer by @Daniel Lopez.

Tests. Most crucially, $t$ tests can work quite well even if the approximation to normal distributions is rough. You're showing the distribution of one group, but not the other, and have not explained whether the groups are paired or not. If paired, then it's the distribution of the differences that is germane.

Either way, I'd wager -- given what you show -- that the results of a $t$ test will be consistent with those of a (Wilcoxon-)Mann-Whitney test, while delivering also concrete results such as a confidence interval for the difference of means, which is much more likely than a significance test to be scientifically or practically interesting and useful.

There is no great harm in doing both tests.

Transformations. I am a big fan in general but on the evidence of your graph -- we need more evidence -- it is possibly not worth the complication. If you jump the other way, then cube roots preserve sign while pulling in tails, so that negative, zero and positive values remain that way. In practice, your software will need to be told how to handle varying sign with some incantation such as sign(x) * abs(x)^(1/3). That said, a transformation such as cube root is not very common, and it's likely that colleagues in your science will react with puzzlement or even hostility to a transformation they have not heard of. (Many scientists combine enthusiasm for new methods for producing data with resistance to any method for analysing data they didn't learn before early graduate school, and I am not a statistician saying that.)

Although the evidence you give does not indicate to me that a transformation is imperative, I would like to comment on an all too common prejudice against transformations. Choosing a transformation isn't changing your data: it is just looking at them on a different scale. Any scientist who has ever used pH, or decibels, or the Richter scale, for example, has already used a transformed scale as a matter of convention or convenience. (All of those examples are logarithmic scales.) Any one who has looked at an atlas has seen different projections in operation, one often being (much) better than another, depending on purpose and the area being shown. Choosing a projection does not change or deny the shape of the planet, which remains exactly as it was before you chose a map to help your thinking. So also with transformations....

Graphics. Histograms are helpful, but by far the most useful graphical way to assess normality, or the degree of lack thereof, is a normal quantile plot, historically often called a normal probability plot or normal scores plot. As mentioned, if your groups are paired, then such a normal quantile plot for differences is advisable. I would put less stress than you do on the apparent outliers: slight positive skewness is par for the course in many fields and the highest values can be perfectly natural, more akin to basketball players than invading giants who are threatening havoc.

Given your data, I would want first to go beyond histograms. Many threads here are pertinent. These links will suggest others:

How to visualize independent two sample t-test?

Interpreting QQplot - Is there any rule of thumb to decide for non-normality?

Note. You have hundreds of measurements, it seems. Even trivial differences will qualify as significant at conventional levels. The real question is the magnitude of the differences, where Wilcoxon-Mann-Whitney cannot help you.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
  • Nice point of view. But I can see how colleagues in the medical field for example might not be very happy if you make claims like "The differences in the cube roots of expression levels of gene X between patients and control groups were significant". You would have to provide a justification (possibly in terms of medical expert knowledge) of why cubic roots of the measurements are more relevant than the raw measurements. – Daniel López Mar 29 '19 at 15:42
  • I've made the same point in my answer. Anyone asking why cube roots would have a good question and the data analyst would need a good answer. On the limited evidence in this question (one histogram) and given the sample size I'd expect $t$ tests to work well on the untransformed data. In any case, the analyst wielding Mann-Whitney would have to explain that too. – Nick Cox Mar 29 '19 at 17:57
  • This is a very interesting second opinion, and an answer that got me thinking again. I guess in statistics there often is no clear right and wrong. Clearly there are things which are wrong, but often there are also several valid approaches and points of view. I will read more about transformations. Btw, the 2 groups are not paired (and I am aware of the importance of effect sizes, but thanks nevertheless for mentioning it, as it is very important and maybe others should read it here.) – John Silver Apr 01 '19 at 09:15
  • My guess remains that a transformation is not needed in your case. My answer was in part directed against the first answer with its prejudice against transformations. – Nick Cox Apr 01 '19 at 09:24
  • @NickCox OK, I can post the data for the 2 groups, but I am not sure whether that alone would help, as one needs to covariate for various demographics measures. And I am not sure whether I am allowed to post these as well, but I doubt that. I will check that. – John Silver Apr 01 '19 at 09:30
1

It seems to me that in your case, the most appropriate test would be the Mann-Whitney-U test, which does not assume the normality of data (non-parametric) and works for unpaired samples.

For these cases, I really like the STAC Web Platform, because it has an easy to follow assistant which helps you in selecting the most appropriate statistical test.

Regarding your idea of applying some transformation to data to make it closer to a normal distribution, I don't see how that can be a good idea... Why would you artificially modify the natural distribution of data, if what you want is to draw conclusions from it to learn about a real world phenomenon. If the distribution of your observations is not normal, that is a property of the real world phenomenon you are studying and I believe altering that won't help you. But maybe someone can provide a different argument in this regard.

A possible exception to this might be the case when you suspect that the underlying distribution of data is indeed normal, but you have some outliers that are skewing the distribution. If you have reasons to believe that some of your measurements are outliers in the sense that they might be the result of measurement errors, and do not carry useful information about the real world phenomenon you are studying, then I believe you could use some outlier detection method to discard those measurements, as a previous step to statistical testing. For example, take a look at Grubbs's test.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
Daniel López
  • 5,164
  • 2
  • 21
  • 42
  • Yes, I agree that Mann-Whitney would be the most suitable test. But what about the data transformation option? Would it be valid to transform the data and then apply a test? A colleague suggested it to me, but I wonder whether it is legit and what the pros and cons are. – John Silver Mar 29 '19 at 14:15
  • I have added my opinion about that to the answer. I have honestly never seen someone using such an approach. Besides, it is not clear how would you do that without losing information or inventing new data... – Daniel López Mar 29 '19 at 14:22
  • 1
    Thanks a lot for the explanation you added. You confirmed what I had a vague feeling about: by transforming the data, you are clearly changing it. And that makes sense only with a very good reason to do so. – John Silver Mar 29 '19 at 14:50
  • 1
    Your resistance to transformations is thoughtful but would stop you (e.g.) ever enjoying the benefits when a log transformation simplifies data dramatically. – Nick Cox Mar 29 '19 at 15:17
  • It depends... If you are trying to build a predictive model for example, I would go with any transformation that improves the generalization performance. But when it comes to drawing conclusions about things like medical measurements... I don't see it so clear. – Daniel López Mar 29 '19 at 15:46
  • This is more about drawing conclusions about medical measurements. – John Silver Apr 01 '19 at 10:10