I have a sample of 5 numbers from known interval [0, 10]. Is 5 numbers is enough to make some conclusions about whether these numbers are drawn from uniform distribution or not?
Asked
Active
Viewed 1,992 times
1 Answers
6
To test $U(0,10)$ with $n=5$?
Not with much reliability, unless it's extremely nonuniform.
If you're expecting U-shaped alternatives, you might consider say an Anderson-Darling test, (it usually shows up well in power studies, so it's a good one to keep in mind more generally) -- but the power at $n=5$ is going to be quite low.
It's substantially more powerful than the Kolmogorov-Smirnov test on U-shaped alternatives.
With hill-shaped alternatives (like a beta(3,3), say), at n=5 the Anderson-Darling is very biased (i.e. has much less than $\alpha$ probability to reject), so it's not a great choice there, and the Kolmogorov-Smirnov test does better (though it, too, is biased - which means you have more power against a beta(3,3) by rolling a 20 sided die and rejecting when it comes up with a '1').
Here's an idea of how serious the bias problem is at n=5 (here I used a significance level of 10%, since $n$ is so small):
With a skew alternative that "piles up" probability at one end (such as a beta(2,1)), the power of the two tests is reasonable - somewhat like the power for the U-shape case, but the power of the two is more similar.
With those problems in mind, here are some followup questions:
Why are you testing uniformity?
Do you have any sense of likely alternatives?
(If you can narrow down the alternatives, you may be able to construct a test that has at least a little bit of power.)
What are the relative costs of the two types of error?
Glen_b
- 257,508
- 32
- 553
- 939
-
Good answer. @Glen_b what did you use to generate the graphic? – Drew75 Mar 04 '14 at 05:51
-
@Drew75 The plot (and all the power calculations) were done in R. The little densities in the bottom margin were generated separately and pasted into the image (basically out of laziness, you can actually do it all in R), as well as minor reformatting (like the note about test bias and the marginal marking of the significance and a couple of other tweaks were added by editing the image). – Glen_b Mar 04 '14 at 06:13
-
Excellent. I suspected R, but I was most curious about how you put the density graphs on the x-axis. – Drew75 Mar 04 '14 at 06:15
-
@Glen_b: I know that A-D can be used to show that it *does not* follow a uniform distribution (i.e. reject $H_0$ if $p > 0.05$). How can we show that it *does* follow a uniform distribution? – Jacob May 05 '14 at 20:39
-
@Jacob - you simply *can't* show that. Consider that a non-uniform distribution may be arbitrarily close to a uniform. Even at huge samples the best you can do is say that it is in some sense close to uniform. – Glen_b May 05 '14 at 22:49
-
@Glen_b: I see. So how does one usually validate their models? – Jacob May 06 '14 at 14:32
-
@Glen_b: I guess what I'm asking for is an equivalence test for this ; is that possible? – Jacob May 06 '14 at 15:33
-
One can see whether a model is *plausible*, not whether it's exactly true. Something akin to an equivalence test may be possible, but there are some issues to address in the case of goodness of fit (since there are so many more ways to differ than with a parameter shift). It would make a good question to post. – Glen_b May 07 '14 at 02:07