3

If I have a coin that produces head with probability p, I know there are several methods to estimate a confidence interval for p with a given confidence level.

If I also know that the coin was randomly selected from a set of coins whose distribution of p is known, I think that it is possible to improve the quality of the estimation.

I tried to find if this is a known problem and if there are known solutions, but I found nothing.

Is there anyone who knows if this is a studied problem or has any idea on how it could be possible to leverage the known distribution of p in order to improve its estimation?

maxvv
  • 51
  • 5
  • If you know p, then you don't need to estimate it and you don't noted confidence intervals for it. – Tim Jul 28 '17 at 05:35
  • 2
    what you describe is a mixed Binomial distribution: it is like a two step approach: (1) you daw a $p$ from a distribution and then (2) given the drawn value you draw from a Binomial with that success probability. You should google for mixed binomial or for lexian distribution. –  Jul 28 '17 at 05:45
  • It very much depends on what your distribution for $p$ is. Note that in general having variability in $p$ will likely lead to wider confidence intervals or less certainty (in a bayesian framework). – jds Jul 28 '17 at 16:41
  • 1
    Tim, that's true, but I don't know p, I can just toss the coin and make observations in order to estimate p. fcop, thanks for the tip. I think I have found what I need thanks to your comment: https://www.jstor.org/stable/2984516?seq=1#page_scan_tab_contents. jds, thanks for the observation. – maxvv Jul 28 '17 at 16:49
  • So what do you mean by knowing the distribution of p? If you know the distribution, you know everything: mean, mode, variance, quantiles etc. – Tim Jul 28 '17 at 22:44
  • 1
    @Tim everything except the particular value of $p$ that was drawn when you started observing heads.... – Glen_b Jul 30 '17 at 10:07

2 Answers2

6

I don't think either of the comments under the question quite get at what's going on here.

My understanding of it is this (and please correct me if I am wrong): a specific value for $p$ is drawn from a known distribution, but we don't observe that $p$. Instead we observe a binomial random variable with that $p$. You want to use both the information in the known distribution for $p$ as well as in the likelihood to make some inference about $p$.

On that understanding:

A random $p$ obtained from a known distribution plus some information about that specific draw of $p$ from a sample is naturally answered with Bayesian approach; the known distribution of $p$ is the prior, which you then update via the sample (specifically the posterior is proportional to the product of the likelihood and the prior).

$$f(p|X)\,\propto\, f(X|p)\,f(p)$$

For example, if the prior were beta, the posterior would also be beta (since the beta is conjugate for a binomial likelihood).

If the prior were symmetric triangular, the posterior would be piecewise beta (with two continuous pieces).

More generally we could use a variety of techniques to obtain the updated information about $p$. On this simple problem - assuming a prior that didn't work nicely with the likelihood - one option would use numerical integration to scale the product of the prior and the likelihood to a density.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • I don't think the question is trying to get at bayesian posteriors. I think @maxvv is trying to ask about hierarchical/compound distributions. This can be very similar to the answer you provided but could also be a frequentist question. In either way, unless $f(p)$ has a specific form, simulation may be his best bet. – jds Jul 28 '17 at 16:40
  • How are you going to get a confidence interval for $p$ out of an estimate of the posterior distribution of $p$? You can get a credible interval, to be sure, but that's a different animal. – Kodiologist Jul 28 '17 at 16:42
  • Thanks Glen, I was suspecting that the Bayesian approach was the way to follow, but I am not very familiar with statistics. I have +1 you answer, but I am not reputable enough to have my vote displayed. :) – maxvv Jul 28 '17 at 16:47
  • @jds the answers could be quite different so I think it's important to try to figure out what problem is being solved. If the question were about the distribution of the number of heads, then it would be a problem of compound distributions, as you say. But the question is about inference on $p$. I don't quite see how looking at that compound distribution for the number of heads gets you to inference on $p$. If you can clarify for me how you see that working, I'd be interested to see if I could improve this answer – Glen_b Jul 30 '17 at 01:11
  • 1
    @Kodi You're correct that a credible interval for $p$ is not a confidence interval; I wasn't assuming the OP was absolutely fixed on having an interval with exactly the properties of a confidence interval (like the coverage guarantee) but I should still have mentioned the difference. – Glen_b Jul 30 '17 at 01:13
2

Thanks everybody for the help. I am indeed in an easy situation, where I can approximate my prior distribution with a beta distribution

Beta($\alpha_0$ ,$\beta_0$).

Therefore, the posterior distribution is

Beta($\alpha_0$ + observed_heads , $\beta_0$ + observed_tails).

I found an answer to another question that uses a very nice example that can be applied to my case: https://stats.stackexchange.com/a/47782/44624

maxvv
  • 51
  • 5