4

I think I am dealing with this issue http://www.r-bloggers.com/normality-tests-don%E2%80%99t-do-what-you-think-they-do/

I have large data set(10k data points) that slightly diverges from normal, and I get p-value of 0. I am interested in having perhaps a more crude test that tells me if data is extremely divergent from normal, versus looking somewhat normally distributed. I am currently trying Kolmogorov-Smirnoff, and in both cases I just get p-value of 0. Any alternatives?

I also looked at this: Is normality testing 'essentially useless'?

So, taking all this into consideration, is there any kind of test I can perform that distinguishes between data that's roughly normal, and not normally distributed at all?

I am using scipy.

Community
  • 1
The Baron
  • 611
  • 1
  • 6
  • 16
  • 2
    One of the very best ways is to look closely at the probability plot, because it will show you precisely *how* the data "diverge" from normal. – whuber Nov 25 '15 at 19:31
  • Sure, but I want to output some sort of summary statistic as well(this is part of an automatized process that should not involve human intervention), – The Baron Nov 25 '15 at 19:32
  • 1
    Because you have already applied some kind of test, what's the matter with reporting *its* test statistic? – whuber Nov 25 '15 at 19:36
  • Because it is always 0, even when the data is approximately normal. Please see the links I have provided. I want the test to actually make a difference, and not be 0 all the time. – The Baron Nov 25 '15 at 19:36
  • 2
    The p-value is not the test statistic. Any distribution test, such as the Kolmogorov-Smirnov, will provide you a statistic that measures how far the data depart from a random sample of a Normal distribution. It seems that you're asking us to propose a bad test, but you haven't provided any information about exactly how bad it needs to be. That's going to be difficult to interpret and answer. – whuber Nov 25 '15 at 19:39
  • Thanks for your input. Is there any kind of test for 'approximate normality' (that you probably call bad test) you would recommend? – The Baron Nov 25 '15 at 19:41
  • 2
    I'd always support use of probability plots here; and also the view that **the probability plot itself is a test statistic**. But it's better to measure non-normality, as that sounds like a better goal for your tastes, than to focus on P-values. (BTW, the P-value may be reported as 0, but it's just very small, not 0.) Classical measures are skewness and kurtosis; better measures in many ways are their L-moments equivalents. Many threads on this forum on both. – Nick Cox Nov 25 '15 at 19:45
  • A normal probability plot will show that. Also a kernel density plot with reference normal superimposed. The best test statistics need not be single scalars; nor can a scalar catch everything that might be interesting or important about a distribution. (An outlier is usually an individual data point, not an odd distribution.) – Nick Cox Nov 25 '15 at 20:18

1 Answers1

4

a more crude test that tells me if data is extremely divergent from normal

You very likely don't want hypothesis tests at all, since they don't answer that question.

You're trying to answer a question related to "effect size" ("how far from normal is it?" is an effect-size type question) but hypothesis tests don't answer that question.

Some goodness of fit statistics (as whuber suggested in comments) do give a measure of "how far from normal"..., as does looking at say a Q-Q plot, like Nick Cox mentioned, but I bet you have a more specific question than would not necessarily be well answered by using some measure essentially at random.

For example, you might perhaps have an underlying question nearer to "How badly could this particular kind of inference I wish to use be affected by the kind of (and degree of) non-normality of the distribution from which my data were drawn?"

[The answer to that depends on what you're trying to do! Different forms of inference have may be sensitive to different aspects of the distribution, and to differing degrees. More detail would be necessary before useful advice could be given for a question like that.]

You may have some other kind of question of course, but how to measure impact on whatever you're trying to do will again depend on what you're trying to achieve.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • I am dealing with measuring Pearson's correlation, for the purpose of evaluating feature quality. This is basically an OLS on 1 variable, Sometimes I get reasonably high correlation(10%), but when I look at the plot, it is completely bogus. After some thought, I realized that those scenarios violate most of the OLS assumptions. So, I want to catch these cases automatically, so I can know that the obtained correlation is not valuable. – The Baron Nov 25 '15 at 23:24
  • The "bogus-ness" of what you're trying to do probably relates more to other things than normality. What were you actually testing for normality? (Alternatively, what is it you think needs to be normal in OLS?) – Glen_b Nov 25 '15 at 23:34
  • I am looking at the errors. If I am to correctly evaluate a signal(feature), and try the correlation I obtain, I also want the errors to be normal, when doing the OLS of the target on that feature. Is this right? – The Baron Nov 26 '15 at 17:16
  • You need something more or less near normality if you are performing inference that relies on it (like hypothesis tests or confidence intervals based on normal assumptions) – Glen_b Nov 26 '15 at 18:19