Interpreting results of prop.test()

Question

Suppose that I have two approaches to a particular problem. Approach A is observed to succeed 685 times out of 1347 attempts. Approach B is observed to succeed 2100 times out of 3748 attempts. I want to see if Approach B is preferable to Approach A.

In R I run:

prop.test(c(2100,685), c(3748,1347), alternative="greater", correct=FALSE)

and I get:

data:  c(2100, 685) out of c(3748, 1347)
X-squared = 10.7124, df = 1, p-value = 0.0005321
alternative hypothesis: greater
95 percent confidence interval:
0.02568765 1.00000000
sample estimates:
  prop 1    prop 2
0.5602988 0.5085375

I have the following questions.

1. Let $p_A$ (resp. $p_B$) be the proportion of successes of Approach A (resp. B). Implicitly, my null hypothesis is that $p_A=p_B$, right? And then, since the $p$-value is less than the significance level, I reject the null hypothesis. To me this seems that I am led to conclude that $p_A \neq p_B$. Of course, it seems likely by just looking at the data that $p_B>p_A$.

2. Is the syntax alternative="greater" included to rule out $p_A>p_B$?

3. Similarly, does the line about the confidence interval mean that the "true" value of $p_B-p_A$ lies in the interval $(.02568765,1)$?

Answer 1

This issue isn't really specific to prop.test(), the issue is more generally about the nature of one-sided hypothesis tests. In light of that, you should probably read @whuber's answer here: Justification of one-tailed hypothesis testing.

1. The gist of the issue is that your null encompasses all possible differences less than or equal to $H_0\!: p_A = p_B$. In order to rule them all out, you need to rule out the most reasonable possibility, namely: $H_0\!: p_A = p_B$. If $p_B$ is sufficiently greater than $p_A$ that $H_0\!: p_A = p_B$ is no longer believable, then $H_0\!: p_A > p_B$ is not a reasonable possibility either.

2. Yes, the syntax alternative="greater" rules out $H_0\!: p_A > p_B$ as well. Relatedly, it also causes R to only use one tail of the null distribution to assess the p-value.

3. Yes, the confidence interval refers to the "true" value of $p_B-p_A$.