compare weighted coins

Question

I have n (much larger than 2) coins that can give Head or Tail in a toss. For each coin, I have one sequence of trials which may have different lengths. For example, a conceptualized dataset could look like: Coin 1: HHT Coin 2: HHTTHHTHTTHHHTTHTHHHHH Coin 3: THHHTTHHTTTTTTT

I would like to test whether all coins are identical, i.e., whether they have the same probability of giving Head or Tail. How would you approach this problem? Thank you.

Could you indicate why the trial lengths vary? If the length of a sequence depends in any way on its outcomes, that ought to affect the analysis. — whuber, Mar 28 '17 at 23:04
Assuming this is the real data I do not think that there is enough evidence to say with any confidence that any of the three coins are biased. The larger the sample size the greater the confidence. — Michael R. Chernick, Mar 28 '17 at 23:06
This is not the real data. There is no reason for the sequence lengths to be different, let's simply assume that the experimenters performed different number of coin tosses for each coin. — Andrea, Mar 28 '17 at 23:39

score 0 · Answer 1 · answered Mar 29 '17 at 03:05

You could perform a $C$ likelihood ratio tests, where $C$ is the number of coins.

Let $L(\theta|y) = \prod_{c\neq c_j} \operatorname{Binomial}(y_c; n_c,\theta_c)\cdot\operatorname{Binomial}(y_{c_j}; n_{c_j},\theta_{c_j})$. We can test $H_0: \mbox{all $c_i$ are equal}$ vs. $H_j:c_i=c_{i'},\; \forall i,i' \neq j$, by finding the tail probability of the test statistic

$$T_j(y) = -2(\log L(\hat{\theta}_0|y) - \log L(\hat{\theta}_j|y)) $$

assuming that it follows a $\chi^2_1$ distribution. Here $\hat{\theta}_0$ maximizes $L(\theta|y)$ with the constraint of $H_0$ and $\hat{\theta}_j$ maximizes over the constraint of $H_j$. Since there are possibly a lot of p-values, you'll want to adjust them for multiple comparisons, e.g. using Bonferroni.

Intuitively, this is a way to see if one coin is behaving differently than the rest.

This approach is problematic because a Bonferroni correction is far too severe due to the correlations among the results. — whuber, Mar 29 '17 at 22:19

score 0 · Answer 2 · edited Apr 13 '17 at 12:44

0

I think the Tukey–Kramer method is appropriate here. Implemented in R as TukeyHSD. Although it does make a normality assumption, it should be fine as long as your samples are larger than about 20 or so for each coin. If the means are reasonably close together, the homogeneity of variance assumption will roughly hold as well.

Quoting wikipedia:

When doing all pairwise comparisons, this method is considered the best available when confidence intervals are needed or sample sizes are not equal

More details here

edited Apr 13 '17 at 12:44

Community

1

answered Mar 29 '17 at 23:14

AaronDefazio

1,551
7
11

This is a good answer that includes the appropriate caveats. Just want to make sure that it is clear to the original poster that the example provided by the OP, with a sequence of only 3 tosses, is not amenable to this method. If the dataset has **any** sequences this short, this method can fail spectacularly. – Jacob Socolar Mar 30 '17 at 04:18

Jacob Socolar · Answer 3 · 2017-03-30T04:16:50.410

Especially if some of the coin sequences are relatively short (so it is important to model the number of heads from a binomial rather than normal distribution), one option is to fit a random-effects binomial model to the data (i.e. with a random effect of coin), and to compare it to a model without the random effect using a likelihood ratio test.

Edited to add: If the likelihood ratio test supports the random effects model, then it is appropriate to conclude that the coins are not identical.

compare weighted coins

3 Answers3