0

I have 3 measurements of a very skew distribution, depending on three different technique. In general, I can do thousands of test and the result is either "fail" or "success", being the "fail" far more common than the success (more than 99% of fails). So, I have these data:

Random: Ntests = 20894 Nsuccess = 26 percentage = 0.124438% Normal: Ntests = 229848 Nsuccess = 334 percentage = 0.145313% Optimized: Ntests = 20272 Nsuccess = 44 percentage = 0.217048%

How can I check that the differences in the percentages are significant?

user1753235
  • 1

1 Answers1

0

The appropriate statistical test is Fisher's exact test. Given your large numbers, you will need to use simulation to obtain $p$ values. In R, you can do this as follows:

> TT <- rbind(c(20894-26,229848-334,20272-44),c(26,334,44))
> fisher.test(TT,simulate.p.value=TRUE,B=10000)

        Fisher's Exact Test for Count Data with simulated p-value (based on 10000 replicates)

data:  TT
p-value = 0.0282
alternative hypothesis: two.sided

See Test if two binomial distributions are statistically different from each other, in particular David Makovoz' answer, which points to Fisher's test, which works for more than two groups.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357