0

I have a typical dataset: sample A (method A) which has 200 samples and 25 failures. sample B (method B) which has 240 samples and 0-10 failures. Note that 0 failures is possible.

Null hypothesis is that the proportions are the same with 5 % significance level. Alternative hypothesis is that method B is the better and reduces failures. I want to reject the hypothesis. This is a one-tailed situation. I planned to use the basic z-test for comparing proportions.

Many Stats books say, however, ...there should be at least 10 failures and successes in both samples if one wants to use the z-test. This condition is obviously not fulfilled here because of sample B. So z-test can't be used, right?

So:

Q1: How to bypass this restriction of minimum number of failures and successes?

Q2: What alternative test should I use to reject the null hypothesis?

(Using Numpy/Scipy or even MatLab is totally fine here. The data will be processed with a computer.)

Bonus question: What is the mathematical reason for the minimum requirement? If sample A has 10 failures and sample B has 10 failures we can reject the null hypothesis. But if sample B has even less failures we can't. This feels unintuitive.

Paapaa
  • 101
  • 2
    Please see https://stats.stackexchange.com/search?q=binomial+hypothesis for a great many accounts of binomial hypothesis testing. – whuber Jul 12 '19 at 11:51
  • It really says 10 failures? Not 10%, but 10 failures? So two heads in eight flips can’t be compared to six heads in eight flips? Reference? – Dave Jul 12 '19 at 11:54
  • Here is a direct quote: "The number with the trait or response of interest and the number without the trait or response of interest is at least 10 in each sample." This is from the book "Mind on Statistics" by Utts & Heckard, 5th edition. – Paapaa Jul 12 '19 at 12:05
  • Thanks for the links. I'll see if Fischer's exact test is what I'm looking for. – Paapaa Jul 12 '19 at 12:12

0 Answers0