1

I would like to parameterize a unimodal beta distribution from samples of a Bernoulli random variable. I'd strongly prefer a method that simply defines $\alpha$ and $\beta$ analytically as functions of the sample mean and variance or sample quantiles, rather than an iterative method. Everything I've found on the method of moments results in u-shaped (bi-modal) parameterizations, i.e. $\alpha < 1$ and $\beta < 1$. My attempts to solve the system of equations has run into the algebraic weeds. What is the right way to do this? It seems that there should be a way via sample quantiles (see this thesis, this question, and this one).

Gregory
  • 113
  • 4
  • 3
    Can you clarify your situation? Why are you trying to do this? What does it mean to "parameterize a unimodal beta distribution from samples of a Bernoulli random variable"? Is this something like an empirical Bayesian approach, & you want to determine what the prior should be based on a different dataset? – gung - Reinstate Monica Apr 05 '17 at 21:25
  • There is only one specific value for the parameters $\alpha$ and $\beta$ that makes the beta unimodal with the mode at 1/2. – Michael R. Chernick Apr 05 '17 at 21:29
  • 2
    @MichaelChernick - I would have thought all values of $\alpha=\beta\gt 1$ would make the Beta distribution unimodal with a mode at $\frac12$ – Henry Apr 05 '17 at 21:32
  • 2
    I assume you are looking for a conjugate prior for the probability parameter of your Bernouilli random variable. If your sample of the Bernoulli random variable produces $x$ successes out of $n$ attempts, then $\alpha= x+k$ and $\beta=n-x+k$ might be an option for some non-negative constant $k$. If $k\ge 1$ or both $x \ge 1$ and $n-x \ge 1$ then this will be unimodal (strictly unimodal if the inequalities are also strict) – Henry Apr 05 '17 at 21:37
  • @Henry If you look at t he wikipedia article "Beta density" it shows quite clearly that $\alpha$=$\beta$=2 is the only density that is symmetric about 1/2 with a mode at 1/2. – Michael R. Chernick Apr 05 '17 at 21:42
  • 2
    @MichaelChernick all parameterizations for which $\alpha = \beta$ are symmetric about 1/2. With that constraint, for $\alpha > 1, \beta > 1$ there is a single mode at 1/2. As the parameters approach 1 dispersion increases until the mode disapears at $\alpha = \beta = 1$. – Gregory Apr 05 '17 at 21:58
  • @gung I am trying to create an informative prior for a Bernoulli. – Gregory Apr 05 '17 at 22:01
  • @Gregory The only definition of mode that I am aware of for a probability density is that it is the largest value. The only other density I saw symmetric about 1/2 has its smallest value at 1/2. – Michael R. Chernick Apr 05 '17 at 22:03
  • @MichaelChernick, that's just a function of the Beta distributions they decided to illustrate on the Wikipedia page. Henry is right: all Beta distributions w/ a=b will be unimodal about .5. – gung - Reinstate Monica Apr 05 '17 at 22:11
  • @Henry That's definitely got me in the right direction. Might be the answer. – Gregory Apr 05 '17 at 22:15
  • @gung The article gave one example where the two parameters are equal. It is symmetric and U shaped. So that can't apply in all cases. It is a counterexample. – Michael R. Chernick Apr 05 '17 at 22:21
  • @MichaelChernick, you're right, I misspoke: It is a=b & a>1. Any Beta distribution whose parameters meet those criteria will be unimodal & centered on .5. – gung - Reinstate Monica Apr 05 '17 at 22:35
  • @gung If you replace "unimodal" with "symmetric" I can agree with you. The U shaped symmetric one in the wikipedia article actually has two equal modes at 0 and 1. – Michael R. Chernick Apr 05 '17 at 22:39
  • @MichaelChernick, consider [this figure](http://zoonek2.free.fr/UNIX/48_R/g616.png) (I just googled it). The black Beta distribution is a=b=10. It is clearly unimodal & centered on .5. – gung - Reinstate Monica Apr 05 '17 at 22:43
  • @Henry, given the OP's encouragement, why not turn that into an official answer? It might be nice if you could give some thoughts about how to choose an appropriate k. – gung - Reinstate Monica Apr 05 '17 at 22:44
  • @gung It was the OP, Gregory who gave the right answer and not Henry. I think it is fine for Gregory to give his own answer. – Michael R. Chernick Apr 05 '17 at 22:49
  • @Henry, I found a more general version of what you proposed, which explains how to interpret your $k$. Your lead got me there, so I don't mind crediting you with the answer if you want to write it up. Otherwise I'll do it. The key is in Brown, Lawrence D., T. Tony Cai, and Anirban DasGupta. "Interval estimation for a binomial proportion." Statistical science (2001): 101-117. https://www.jstor.org/stable/2676784. – Gregory Apr 05 '17 at 23:21

1 Answers1

3

Repeating and developing my comment

I assume you are looking for a conjugate prior for the probability parameter of your Bernoulli random variable. If your sample of the Bernoulli random variable produces $x$ successes out of $n$ attempts, then $α=x+c$ and $β=n−x+k$ might be an option for some non-negative constants $c$ and $k$. Unless you have other prior information, it would be natural to take $c=k$

So long as at least one of $\alpha$ and $\beta$ is greater than or equal to $1$, you will have unimodal distribution (though uniform if $\alpha=\beta=1$, and a distribution concentrated on a single point if $\alpha$ or $\beta$ is $0$). So your posterior distribution after at least one sample will be unimodal no matter what $c$ and $k$ you choose; if you want the prior distribution to be unimodal too and symmetric, then choose $c=k\gt 1$.

For other people less worried about the shape of the prior distribution, common choices which have some sort of "uniformative" rationalisation are $c=k=0$ (giving $\alpha=x$ and $\beta=n-x$ so an unbiased posterior expected value of $\frac{x}n$), $c=k=\frac12$ (the Jeffreys prior) or $c=k=1$ (the uniform prior)

Henry
  • 30,848
  • 1
  • 63
  • 107