Can we compare the difference of two specific proportions in a 3x3 contingency table?

Question

I have a 3x3 contingency table and would like to compare the the specific proportion's relationship.

For example, can we test if the proportion of people who give response A in age group 1 is higher than that in age group 3?

Answer 1

The null hypothesis of independence is strongly rejected by a chi-squared test:

a = c(13,25,28); b = c(21,31,24); c = c(21,20,4)
TBL = rbind(a,b,c);  TBL
  [,1] [,2] [,3]
a   13   25   28
b   21   31   24
c   21   20    4
colSums(TBL)
[1] 55 76 56
chisq.test(TBL)

        Pearson's Chi-squared test

data:  TBL
X-squared = 17.202, df = 4, p-value = 0.001766

An ad hoc prop.test in R, of the null hypothesis that the proportion of As in Age groups 1 and 3 is the same is also strongly rejected, as shown below. However, the proportion in Gp 1 is smaller than the proportion in Gp 3.

prop.test(c(13,28), c(55, 56))

        2-sample test for equality of proportions 
        with continuity correction

data:  c(13, 28) out of c(55, 56)
X-squared = 7.1863, df = 1, p-value = 0.007346
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.45415527 -0.07311746
sample estimates:
   prop 1    prop 2 
0.2363636 0.5000000 

Note: Even if this second test is your main goal, it would be not be appropriate to do that test until first testing whether the overall table is consistent with independence of Age and Response. Suppose Age and Response are independent, but you notice a discrepancy in counts, which you test and find significant. That could be just by chance, and you will have made a 'false discovery'.

For a simpler example: Suppose you have six age categories, 30 subjects per category, with possible responses Yes and No. Actually, all six categories have the same probability $(1/6)$ of Yes answers. But Yes counts happen to be $4, 3, 9, 4, 6, 2.$ You notice that $9$ and $2$ are quite different, and prop.test does find a significant difference. That means nothing. You have hand-picked the extremes. Doing this, you will often find bogus "differences."

set.seed(1212)
rbinom(6,30,1/6)  # All six categories actually same
[1] 4 3 9 4 6 2   # Happen to get min 2 and max 9
prop.test(c(9,2),c(30,30))  # Bogus test of extremes

        2-sample test for equality of proportions 
        with continuity correction

data:  c(9, 2) out of c(30, 30)
X-squared = 4.0074, df = 1, p-value = 0.0453  # FALSE DISCOVERY
alternative hypothesis: two.sided
95 percent confidence interval:
 0.01329788 0.45336879
sample estimates:
    prop 1     prop 2 
0.30000000 0.06666667 

In this scenario, if Yes and No are equally likely responses for all age groups, then the false discovery rate for testing min against max is about 23%.

set.seed(2020)
pv= replicate(100000, 
     prop.test(range(rbinom(6,30,1/2)),c(30,30))$p.val)
mean(pv < 0.05)
[1] 0.23352