7

Suppose today I'm going to flip a coin. I believe that 9 of 10 flips will come up heads. I flip the coin and 8 of 10 are heads. Is my distribution of belief

  1. beta(9+8, 1+2)
  2. beta(1+9+8, 1+1+2)
  3. beta(m+8, n+2) where I choose m and n based on graphing beta(m,n) and deciding if it feels right. (This is not a facetious answer, it's a real suggestion.)
  4. Hidden option 4.

If I use #1, I believe that is an improper prior, but Wikipedia claims some statisticians use them.

This choice is not obvious to me.

TooTone
  • 3,631
  • 24
  • 33
dfrankow
  • 2,816
  • 6
  • 30
  • 39
  • 5
    If you want your prior to include the information of a *hypothetical sample* of 10 flips out of which 9 came out as heads, then your prior is $\mathrm{Beta}(9,1)$. The mean of the prior is $0.9$. After flipping the coin 10 times out of which 8 are heads, your posterior is $\mathrm{Beta}(17,3)$. The posterior mean is $0.85$. You can use an improper prior, that is no problem. The Beta distribution is the conjugate distribution of the binomial. Check out [this post](http://stats.stackexchange.com/a/58792/21054) for a fully worked example using the Beta distribution. – COOLSerdash Jul 19 '13 at 15:54
  • 4
    Choice #1 is not an improper prior. – QuantIbex Jul 19 '13 at 15:58
  • @QuantIbex Your right, of course (+1). Sorry, my wording was inaccurate: I didn't want to imply that #1 was improper, I just wanted to say that an improper prior is not a problem automatically, at least not mathematically. – COOLSerdash Jul 19 '13 at 16:10
  • 2
    For the Beta distribution, the improper prior is $\lim_{a \to 0}\rm{Beta(a,a)}$. Improper means it isn't a probability density, e.g. doesn't integrate to 1. You might want to consider a non-informative prior, which is flat for all possible values of your parameter: the uniform distribution is suitable here, which is a special case of the Beta distribution, i.e. $\rm{Beta(1,1)}$. See [online beta graphing](http://keisan.casio.com/has10/SpecExec.cgi?id=system/2006/1180573226). – TooTone Jul 19 '13 at 16:10
  • @COOLSerdash In fact, my comment was in response to the OP's statement. Sorry for the confusion. – QuantIbex Jul 19 '13 at 16:16
  • 1
    Improper priors are acceptable so long as they produce proper *posteriors*. For the beta prior with a binomial likelihood, any improper prior will produce a proper posterior provided you observe one (or more) heads and one (or more) tails. – Sycorax Jul 19 '13 at 18:27

1 Answers1

10

If your prior belief is that 9 of the 10 coin flips will come up heads, then you want the expectation of your prior to be 0.9. Given $X \sim \mathrm{Beta}(\alpha,\beta)$ (for conjugacy in the beta-binomial model), then $E[X] = \alpha/(\alpha+\beta) = 0.9$, so you can use this as your first constraint. Obviously this leaves you with an infinite number of possibilities, so we'll need a second constraint which will represent your subjective confidence in this prior belief: the variance of your prior, $\operatorname{var}[X] = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\ $

Decide how you want to set your variance and solve the system of equations for $\alpha$ and $\beta$ to define the parameters for your prior. Justifying your choice of variance here may be difficult: you can always err on the side of a wider (i.e. less informative) variance. The wider you set the variance, the closer your prior will approximate a uniform distribution.

If you want a truly uninformative prior, you should consider using Jeffrey's Prior.

David Marx
  • 6,647
  • 1
  • 25
  • 43
  • 1
    There is no such thing as a [trully uninformative prior](http://stats.stackexchange.com/questions/20520/what-is-an-uninformative-prior-can-we-ever-have-one-with-truly-no-information). – COOLSerdash Jul 19 '13 at 16:54
  • Ok, that's fair. I consider it a 'very effective' uninformative prior, but you're right, I shouldn't be using such rigid/suggestive language. – David Marx Jul 19 '13 at 16:57
  • Thanks for your answer. The gist is: you have to say something about the variance to set the parameters. Seems sensible. I think the human-understandable way to approach it might be "I believe the mean is 0.9, and >0.8 with 95% confidence." It would be interesting to see the easiest way to do that. – dfrankow Jul 19 '13 at 20:13
  • By the way, I could not make heads or tails of Jeffrey's Prior. I'm sure it's very important and useful, but I won't worry about it. – dfrankow Jul 19 '13 at 20:14
  • @dfrankow here's a better explanation: http://www.cs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture7.pdf – David Marx Jul 19 '13 at 23:13