0

I wrote a previous question yesterday which was maybe too long and boring to read. So to try to get an answer, I've boiled down my question to something short and specific which is:

Is there an analytic expression for false-negative rate of binomial tests as a function of number of coinflips and weight of the unknown coin?

That is, if I have a coin that has a chance of $(50-\epsilon)\%$ and $(50+\epsilon)\%$ for heads and tails, what is the chance that I won't get a false-positive for a specific confidence level?

To give a specific example to make it more clear: I have an unfair coin with a 90$\%$ chance of Heads. I can sample my coin by flipping it N times, and want to decide on a value of N that confirms it for a specific confidence level (low false-positive rate). If my p-value that I choose is very small (a high confidence level), then it means without taking a lot of data my false-negative rate is high. That is, I can be looking at my unfair coin data, but my binomial test returns a p-value greater than my confidence level and I cannot conclude my data represents flips from an unfair coin.

Therefore, I want to know how to figure out what my false-negative rate is as a function of number of coinflips and weight of the unknown coin.

This way, if I have a coin of unknown weight, and want to test if it's fair, I can characterize how confident I am that it is not fair as a function of what the coins weight could be. The closer it is to 50/50 the less confident I am that it's correct.

Steven Sagona
  • 385
  • 1
  • 10
  • 3
    If this is a replacement for your previous question, it would be best to paste it over that question as an edit. – whuber Nov 16 '21 at 00:02
  • I have a partly complete answer to your previous question, but it could use more focus. However, several issues in this version concern me; if your test is $H_0:p=\frac12$ (vs $H_1:p\neq\frac12$), then rejecting with a population proportion of $0.5+\epsilon$ would be a true positive not a false positive. If your true null is actually an interval, you should address that (but that situation would perhaps be more typically approached as an equivalence test instead). – Glen_b Nov 16 '21 at 00:51
  • Indeed, the previous version already has at least two distinct formulations of the question which you say are equivalent questions, but which are not. It may be better to focus on a single question, or alternatively perhaps to ask a more general question about how power calculations are typically done (with specific reference to testing fairness of a coin). – Glen_b Nov 16 '21 at 04:48

0 Answers0