0

Problem:

Let $X_1,..., X_{n_1} \stackrel{iid}{\sim} \text{Binomial}(n, p_1)$ and $Y_1,..., Y_{n_2} \stackrel{iid}{\sim} \text{Binomial}(n, p_2)$. I want to test the following hypotheses:

$$ H_0: p_1 = p_2 \\ H_A: p_1 \neq p_2 $$

My approach so far:

Let $\hat{p}_1 = \frac{ \sum_{i=1}^{n_1} X_i/n }{n_1}$ and $\hat{p}_2 = \frac{ \sum_{i=1}^{n_2} Y_i/n }{n_2}$. Test if these proportions are equal using a chi-squared test of homogeneity (see here for details).

My questions:

  1. Is there a better way to do this test? Letting $p_1$ and $p_2$ be the averages of all individual proportions feels silly because it ignores the distributions of the proportions.

  2. Could I let $\hat{p}_1 = (X_1/n, ..., X_{n_1}/n)$ and $\hat{p}_2 = (Y_1/n, ..., Y_{n_2}/n)$ and use a non-parametric ANOVA (like the Kruskall-Wallis test) to test if the vectors of proportions are different? Would this approach be better than my initial proposal?

vivek
  • 111
  • 4
  • See https://stats.stackexchange.com/search?q=two+sample+test+proportion+binomial+score%3A2. – whuber Jun 03 '19 at 19:28
  • 1
    Also see https://stats.stackexchange.com/questions/113602/test-if-two-binomial-distributions-are-statistically-different-from-each-other. – StubbornAtom Jun 03 '19 at 19:57
  • $n_1$ and $n_2$ are the number of samples from each population respectively and $n$ is the number of trials. None of these numbers are necessarily equal to each other. – vivek Jun 03 '19 at 20:13
  • @goatboy3million I see I missed that. – StubbornAtom Jun 03 '19 at 21:25
  • Consider this scenario: You have two methods A & B of teaching how to do a task: $n_1 =50$ subjects are taught using A, $n_2=60$ using B. At the end everybody takes an $n=100$ question True/False test. The first subject taught using A gets $X_1 = 83$ of $n=100$ questions right. That person's score is the binomial proportion $83/100 = .83.$ Altogether in the two groups, you have $n_1+n_2 = 50+60 = 110$ of these binomial proportion scores. Is that anything like your design? Please answer Yes or No, and discuss for clarity. If Y, are there several equiv. versions of the test? Or all exactly same? – BruceET Jun 04 '19 at 01:02
  • @BruceET Yes, that's what my design is. All versions of the test are exactly the same (i.e. every subject takes the same test). – vivek Jun 04 '19 at 12:29
  • OK. Suggest you make details of your Q more exact, as far as possible. About how many questions on exam. Are some quest on the test more important or quite diff than others? Approx values of $n_1$ and $n_2?$ 15? 500? Give a clue what A and B are and how they differ. For your work, what do $p_1$ and $p_2$ actually represent and what would it mean to you if they are significantly different? // If someone wants to answer your Q, they will read _your_ explanation. Don't depend om them reading _my_ long Comment. // If its going to be a chi-sq test, data in table must be _counts_ not proportions. – BruceET Jun 04 '19 at 17:12

0 Answers0