"Reversed" Shapiro–Wilk

Question

The Sharipo-Wilk test, according to wikipedia, tests the null-hypothesis ($H_0$) "The population is normally distributed".

I am looking for a similar normality test with $H_0$ "The population is not normally distributed".

Having such a test, I want to calculate a $p$-value to reject $H_0$ at significance level $\alpha$ iff $p < \alpha$; proving that my population is normally distributed.

Please note that using Sharipo-Wilk test and accepting $H_0$ iff $p > \alpha$ is an incorrect approach since it literally means "we haven't enough evidence to prove that H0 doesn't hold".

Related threads - meaning of $p$-value, is normality testing useless?, but I can't see a solution to my problem.

The questions: Which test should I use? Is it implemented in R?

A null hypothesis of "not normally distributed" is not usable. This space would include all distributions arbitrarily close to, but not quite, normal distributions. You give me any finite set of data. I select the empirical distribution, which is not normal, and therefore belongs to the null space. Cannot reject. — A. Webb, Mar 31 '15 at 16:23
This question, which is the same as your preceding one, asks for the impossible. A proper answer would explain how statistical hypothesis tests work, which is why I pointed you to http://stats.stackexchange.com/questions/31 in a comment to your other question. — whuber, Mar 31 '15 at 16:41
While a null hypothesis "not normally distributed" is impossible, a null hypothesis "is distributed with an absolute values of normal goodness-of-fit statistic that is at least as different as $\varepsilon$" along the lines of an equivalence test seems reasonable. In other words one ought to be able to test against a null of "non-normal by at least *this much*." @gung has suggested precisely this in his answer. — Alexis, Mar 31 '15 at 17:01

score 10 · Accepted Answer · edited Apr 13 '17 at 12:44

There is no such thing as a test that your data are normally distributed. There are only tests that your data are not normally distributed. Thus, there are tests like the Shapiro-Wilk where $H_0\!: \rm normal$ (there are many others), but no tests where the null is that the population is not normal and the alternative hypothesis is that the population is normal.

All you can do is figure out what kind of deviation from normality you care about (e.g., skewness), and how big that deviation would have to be before it bothered you. Then you could test to see if the deviation from perfect normality in your data was less than the critical amount. For more information on the general idea it might help to read my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?

score 5 · Answer 2 · answered Mar 31 '15 at 17:10

I want to calculate a p-value to reject H0 at significance level α iff p<α; proving that my population is normally distributed.

The normal distribution arises when the data is generated by a series of additive iid events (see the quincunx image below). That means no feedbacks and no correlations, does that sound like the process that lead your data? If not, it is probably not normal.

There is the off chance that type of process may be occurring in your case. The closest you can come to "proving" it is to collect enough data to rule out any other distributions that people can come up with (which is probably not practical). Another way is to deduce the normal distribution from some theory along with some other predictions. If the data is consistent with all of them and no one can think of another explanation then that would be good evidence in favor of the normal distribution.

https://en.wikipedia.org/wiki/Bean_machine

Now if you do not expect any specific distribution a priori it may still be reasonable to use the normal distribution to summarize the data, but recognize that this is essentially a choice out of ignorance (https://en.wikipedia.org/wiki/Principle_of_maximum_entropy). In this case you do not want to know whether the population is normally distributed, rather you want to know whether the normal distribution is a reasonable approximation for whatever your next step will be.

In that case you should provide your data (or generated data that is similar) along with a description of what you plan to do with it, then ask "In what ways may assuming normality in this case mislead me?"

I actually know that the data are normal (independent measurement on independent computers), however I need to make some assumption for my thesis.. thank you for clarification and example :) — petrbel, Mar 31 '15 at 17:17
Incidentally, Krieger has provided a lovely critique of the uses of Galton's *Quincunx* in Krieger, N. (2012). Who and what is a “population”? historical debates, current controversies, and implications for understanding “population health” and rectifying health inequities. *The Milbank Quarterly*, 90(4):634–681. — Alexis, Mar 31 '15 at 17:26
@petrbel That situation is subtlety different from what is described above. You can devise a quincunx where each observation is iid but the process that generates the data is not. See here for a log-normal example: [LIMPERT et al. Log-normal Distributions across the Sciences: Keys and Clues. May 2001 / Vol. 51 No. 5. BioScience.](http://stat.ethz.ch/~stahel/lognormal/bioscience.pdf) — Livid, Mar 31 '15 at 17:26
@Alexis I see that Krieger (2012) reproduces the figure from Limpert et al. (2001) and makes the point missed by petrbel: "altering the structure can change outcome probabilities, even for identical objects, thereby creating different population distributions". — Livid, Mar 31 '15 at 17:47

TrynnaDoStat · Answer 3 · 2015-03-31T16:42:11.333

2

You will never be able to "prove" a Normality assumption in your data. Only offer evidence against it as an assumption. The Shapiro-Wilk test is one way to do this and is used all the time to justify the Normality assumption. The reasoning is that you start off by assuming Normality. You then ask, does my data suggest I'm making a silly assumption? So you go ahead and test it with Shapiro-Wilk. If you fail to reject the null hypothesis then the data doesn't suggest you're making a silly assumption.

Notice, people use this similar logic all the time in practice - not just in the context of the Shapiro-Wilk test. They want to use linear regression, look at a $Y, X$ scatterplot and see if linear regression is a silly idea. Or, they assume heteroscedasticity and plot error terms to see if this is a silly idea.

edited Mar 31 '15 at 16:42

answered Mar 31 '15 at 16:24

TrynnaDoStat

7,414
3
23
39

That practice you discribe is exactly the incorrect approach petrbel mentioned. Tests are usually consistent, so the larger the sample size, the larger the probability to declare the normality assumption a silly idea. This is itself silly, because with larger sample sizes, the normality assumption is less critical due to the asymptotic robustness of most procedures. – Horst Grünbusch Mar 31 '15 at 17:25
@HorstGrünbusch Do you disagree that the Shapiro-Wilk test is a valid way to test one's assumption that the data is Normal? – TrynnaDoStat Mar 31 '15 at 17:40
If you agree that it's a valid approach then I'm not sure what you disagree with in my answer. – TrynnaDoStat Mar 31 '15 at 18:02
No. See the arguments here: http://stats.stackexchange.com/questions/2492/is-normality-testing-essentially-useless. Also, it is invalid to test the null hypothesis that two samples have the same variance and to use the Satterthwaite test only if the variances are significantly different and else the t-test for homoskedastic samples. Just simulate this composite procedure on your own: You can yield type-I-error rates of up to $2\alpha$. – Horst Grünbusch Mar 31 '15 at 19:47
@HorstGrünbusch It seems your problem with my answer has to do with the idea of hypothesis testing in general. Specifically, the fact that in many situations hypothesis tests will reject the null with probability 1 as sample size approaches infinity. – TrynnaDoStat Mar 31 '15 at 20:32
Not exactly. The main point is that you fix the type I error rate at the cost of an unknown type II error rate. So if you test the null hypothesis "are the variances equal" and fail to reject, you don't know the actual error probability. That's why failing to reject means we don't know anything. Now I hope it is clear that a test that did not reject does not justify anything. Have you tried the simulation? – Horst Grünbusch Apr 02 '15 at 09:44

"Reversed" Shapiro–Wilk

3 Answers3

Linked