2

I have been studying 2x2 contingency tables and specifically I have been looking at situations where the marginals are fixed by design for one categorical variable.

As an example, suppose a researcher is interested in evaluating whether there is a significant difference in the proportion of male vs female pet owners in a large city. The research performs separate, independent random samples on 500 males and 500 females. The results are summarized in the 2x2 table below.

2x2 contingency table - large sanples

I realize the standard test for homogeneity would be a chi-square test or z-test for difference in proportions (with continuity corrections). When I calculate this I get a p-value of 0.019279. I also performed the Fisher Exact Test (FET) which returns a p-value of 0.019234. Intuitively it makes sense to me that these two tests would produce very nearly the same result, reflecting I think the fact that the hypergeometric distribution converges to a normal distribution for large sample sizes. So far so good....

However, it seems to me that what we really have are two binomial samples, both with sample size of 500. If we are testing the null hypothesis that the proportion of male pet owners is equal to the proportion of female pet owners (vs. alternative that they are not equal), and we further assume that the overall population proportion of pet owners is 0.5 consistent with our sample (which is what a z-test of difference in proportions also assumes), my thinking is that we should be able to calculate the probability of all possible results as the product of 2 binomials, and sum all of these probabilities that are equal to or lower than the probability for the result we obtained from our samples to obtain a p-value. When I do this, I obtain a p-value of 0.056987.

My questions are as follows:

1 Do the p-values I calculate look correct? I have confidence in the calculations for the z-test/chi-square and FET, but the calculation of the p-value using the product of binomials approach I would have expected to also be very close to these other tests so I am concerned there is an error in my calculations (using excel spreadsheet).

  1. Is the product of 2 binomials a valid approach given the circumstances? If so, is it correct to sum all cases where the probability is < or = to the specific probability of obtaining the specific sample results as depicted in the table (as with FET)?

  2. Is it correct to expect that for large samples the p-value should be very nearly equal to the z-test/chi-square test results?

I originally was investigating the product binomial approach for small samples as an alternative to FET, and noticed substantial differences in p-values for many cases. But I had hoped to see convergence of p-values for all these tests for large samples, but results above make me think there is a fundamental flaw in either my thinking or calculations or both!

2x2 contingency table - large samples

user221943
  • 135
  • 1
  • 1
  • 7
  • Could you explain what you mean by the "product of two binomials"? According to the conventional meaning, this would play no apparent role in your problem, leading me to suspect you mean something unusual by this phrase. – whuber May 16 '16 at 16:51
  • Probably using incorrect terminology...what I mean is that in the sample of 500 males we have 231 yesses or "successes", so the probability of this outcome for the males is binomial (n=500, p = 0.5), and for the sample of 500 females with 269 successes we have another binomial (n=500, p=0.5). Since these are independent binomials, the product of these 2 binomial probabilities should be the probability of obtaining the specific results in the table. I use p=0.5 because this is the overall proportion of of pet owners, which is consistent with the assumption made for a z-test of proportions. – user221943 May 16 '16 at 17:58
  • I see: you have obtained the chance of observing *this particular table*. But that's a only a small part of determining a p-value. That's worthy of an answer (+1), but in the meantime you might want to explore our higher-voted threads on the topic, such as http://stats.stackexchange.com/questions/31 . – whuber May 16 '16 at 18:04
  • yes I agree that what I described in my last comment is the probability of the specific result in the table. To arrive at the p-value of 0.056987 as per my original post, I have summed all of the probabilities for all possible outcomes that have the same or lower probability of occurrence as the the probability for the specific outcome in the table. This is analogous the FET but uses binomial product rather than hypergeometric and does not hold the pet owner marginal counts constant – user221943 May 16 '16 at 18:24
  • That's not the correct approach. One doesn't sum lower individual probabilities: one sums probabilities *of events in the critical region for the test,* regardless whether they are greater or smaller than the individual probability of the observed outcome. This (and other) subtleties are what you can find explained in the thread I referenced. – whuber May 16 '16 at 19:28
  • thank you whuber for your continued responses(and patience). I am a bit confused - what outcomes are there that have a higher probability of occurrence that are in the critical region for any hypothesis test? Note that under my approach, as with the fisher exact test, we are not calculating a critical value. We are directly calculating the p-value compared to a significance level. We always have the alternative for discrete distributions of calculating the p-value directly based on the underlying distribution - which in my example is I believe a joint binomial. – user221943 May 17 '16 at 01:11
  • sorry whuber I am just getting used to the site, I know alot but i am wise enough to also realize that i still dont know what i dont know :). Please be patient with me i am just another truth seeker with more questions than can ever be answered. I am being reminded by site monitors not to have extended discussions in comments so this will be my last...do you think you can craft an answer that would pass muster with the site auditors? – user221943 May 17 '16 at 01:26
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/39883/discussion-between-user221943-and-whuber). – user221943 May 17 '16 at 10:39

0 Answers0