3

I have a data set with two groups. I have multiple trials with different Ns in each trial, and different numbers of trials for each group. In each trial, for each group, I am counting the number of successes.

How can I estimate the proportion of successes for each group under a binomial distribution (i.e. success or failure), with confidence intervals? How can I test whether the proportion of successes is the same between the two groups?

The standard Z-test for proportions is only described for one trial per group.

This question wasn't sufficient: Determining statistical significance of difference between two binomial distributions

Nor was this: Test if two binomial distributions are statistically different from each other

Community
  • 1
Dylan Richard Muir
  • 177
  • 11
  • The `N` samples within a single trial are not perfectly independent, but it is fairly reasonable to assume independence. – Dylan Richard Muir Jul 03 '15 at 09:22
  • Do you have the outcomes for each individual in each trial (in each group)? – Daniel Jul 03 '15 at 09:29
  • @Daniel: Sorry if I misunderstand your question. For each of the `N` individual samples in each trial (in each group), I know the outcome -- either success or failure. – Dylan Richard Muir Jul 03 '15 at 09:33
  • But do you know which individual is responsible for each sample in each trial? e.g. if group A has 3 individuals (a, b, c), and the first trial for group A has 5 samples, do you have (1, 0, 1, 1, 0), or ({1, b}, {0, a}, {1, c}, {1, a}, {0, b})? – Daniel Jul 03 '15 at 10:57
  • Individuals are only sampled once, and are not repeated across groups or across trials. Individual ID can therefore be neglected: group A trial 1 is (1, 0, 1, 1, 0). – Dylan Richard Muir Jul 03 '15 at 14:24
  • So the number of successes in each group is the number of successful people and the number of trials is the number of people (for each group again) ? – brumar Jul 06 '15 at 06:41
  • No; for my purposes, a single "trial" consists of successes or failures from several people. There are multiple trials for each group. – Dylan Richard Muir Jul 06 '15 at 06:46

1 Answers1

2

If I'm understanding your question right (and a couple simple assumptions are met), the second link you have gives you what you're looking for. So a simple probability fact is that if X and Y are independent, X~Bin(n,p) and Y~Bin(m,p) then X+Y~Bin(m+n,p). That means that if the trials in each of your groups are independent then

$\sum_{i=1}^{|G_{1}|}X_{i}$ ~Bin($\sum_{i=1}^{|G_{1}|}N_{i}^{1}$, p) and $\sum_{i=1}^{|G_{2}|}Y_{i}$ ~ Bin($\sum_{i=1}^{|G_{2}|}N_{i}^{2}$,p)

where the G's are the groups, the N_i are the n for each binomial trial (superscript tells the group) and |G| is the size of G. From there, you're back at the problem of estimating proportions and confidence intervals in binomial families. The big things to make sure of are that the trials within groups are independent and that you have no reason to believe the probabilities of success are different between trials within groups.

Nick Thieme
  • 1,194
  • 8
  • 11
  • And what if the probability of success differs between trials within groups? For example, if there are differences in how difficult it is to detect "success" between trials? – Dylan Richard Muir Jul 13 '15 at 09:40
  • Using the same kind of logic as above (binomials as sums of bernoulli's) you can treat your sums as poisson binomial's. They're not particularly nice but at least you have a distribution for them. https://en.wikipedia.org/wiki/Poisson_binomial_distribution – Nick Thieme Jul 13 '15 at 14:12
  • Are there any tests that would operate on P.B. distributed sets of values? Or would one have to roll-your-own monte carlo test? – Dylan Richard Muir Jul 13 '15 at 15:01
  • 1
    Here's a link to a better explanation than I could ever give:http://stats.stackexchange.com/questions/93645/hypothesis-testing-on-poisson-binomial-distribution – Nick Thieme Jul 13 '15 at 15:08