4

I am running the following simple Z test for proportions in R:

prop.test(x = c(10, 20), n = c(100, 500))

The results are as follows:

2-sample test for equality of proportions with continuity correction

data:  c(10, 20) out of c(100, 500)
X-squared = 5.1158, df = 1, p-value = 0.02371
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.007256322  0.127256322
sample estimates:
prop 1 prop 2 
  0.10   0.04

How is it possible for the p value of the Z test to be below 0.05, but for the 95% confidence interval to straddle zero? I've found this question on the behaviour of the prop.test function itself and this one on the alternative ways of constructing the CI but I don't think either explains what's going on in this case.

The top answer to this question suggests I'm right in thinking this output should be impossible!

Tom Wagstaff
  • 193
  • 8

1 Answers1

3

I think one difficulty may be in the normal (or chi-squared) approximation for relatively small sample sizes. Also, the 'test' part of the output uses the null hypothesis $p_1 = p_2$ to get a pooled estimate for the standard error, whereas the '95% CI' does not assume $p_1 = p_2.$

Fisher's exact test fails to reject the null hypothesis at the 5% level.

TAB = rbind(c(10,20), c(90,80))
TAB
     [,1] [,2]
[1,]   10   20
[2,]   90   80

fisher.test(TAB)

        Fisher's Exact Test for Count Data

data:  TAB
p-value = 0.07343
alternative hypothesis: 
 true odds ratio is not equal to 1
95 percent confidence interval:
 0.1754757 1.0691978
 sample estimates:
 odds ratio 
  0.4462223

Also, prop.test in R is roughly equivalent to using a chi-squared test on the $2\times 2$ table. With Yates' continuity correction, this procedure does not reject.

chisq.test(TAB)

        Pearson's Chi-squared test 
        with Yates' continuity correction

data:  TAB
X-squared = 3.1765, df = 1, p-value = 0.07471
BruceET
  • 47,896
  • 2
  • 28
  • 76