5

Note: this question has significantly evolved, thanks to inspiring comments by Tim.

Assume there is some "truth" $x\in[0,1]=Beta(1,1)$ that is signaled with some precision. I assume that the resulting posterior distribution (after receiving signal) of $x$ is $Beta(\alpha,\beta)$. To make the signal more explicit, I shall reparametrize the distribution as in this question and this paper (also accounting for uniform prior) with:$\alpha=1+s\phi$, $\beta=1+\phi(1-s)$, where $s\in[0,1]$ is a signal and $\phi$ is explicitly the precision parameter.

Since I know both the prior and the posterior, by Bayes rule, the signal $s$ given $x$ and chosen precision $\phi$ must follow: $$ f(s|x,\phi)=\frac{\Gamma(2+\phi)}{\Gamma(1+s\phi)\Gamma(1+\phi(1-s))}\cdot{x^{s\phi}(1-x)^{\phi(1-s)}}. $$ For $\phi\in\mathbb{N}$ and $s=k/\phi$ for $k\in\mathbb{N}$ this has a nice binomial interpretation: you make $n$ binomial experiments each with success probability $x$ and your signal is equal to the fraction of successes.

What could be an interpretation of a signal in the general case? Is there some intuition? Is this formulation ever used?

Community
  • 1
Joanna F
  • 123
  • 9
  • Why binomial and no other distribution that is "designed" for continuous data? Could you tell us more about your data and $j$ and $k$? I guess it's some kind of amount of signal through some amount of time? – Tim Nov 03 '16 at 09:18
  • There is no data, it is a theory model. I assume that there is some "truth" $x\in[0,1]$ that is signaled with some chosen precision ($k$ in the example above, but a continuous variable would be even better). I had to assume some "signalling technology", and since I operate on $[0,1]$, the beta-binomial distribution seemed to be a reasonable choice. Also, it has a nice interpretation that the signal is $k$ pieces of evidence, each "persuasive" with probability $x$. For technical reasons, a continuous signaling technology might be better - but I want sth tractable. – Joanna F Nov 03 '16 at 09:48
  • Wouldn't you rather want to treat the precision as something like [precision](https://stats.stackexchange.com/questions/183522/antonym-of-variance/183523#183523) parameter, rather then known (static) quantity? From what you're saying it seems that some kind of hierarchical Bayesian model may seem to be better, but it's hard to tell not knowing more details. – Tim Nov 03 '16 at 10:07
  • Yes, in general this is the way I think about precision - more precise signals give posteriors more concentrated around the truth. And it's not static - it is a choice of the person acquiring the signal (who also has to pay the cost $c(k)$). In fact, a Gaussian model was my first choice, but I really want to stick to $[0,1]$. Binomial signals work well in getting more and more concentrated around true $x$, but their discreteness causes some complications in the proofs. – Joanna F Nov 03 '16 at 10:16
  • OK, so what about logistic regression-like hierarchical Bayesian model? You'd transform the linear, continuous "Gaussian" outcome to [0,1] by using logit function. – Tim Nov 03 '16 at 10:21
  • I will Google it :) As I said, since it is a model (meant to give insights, not to describe reality), my priority is tractability and nice intuition, and in this sense beta-binomial did a great job. I just have technical problems with mathematical proofs, especially if $k$ is small. – Joanna F Nov 03 '16 at 10:27
  • You may be also interested in http://stats.stackexchange.com/questions/194182/beta-as-distribution-of-proportions-or-as-continuous-binomial and http://stats.stackexchange.com/questions/67443/does-the-beta-distribution-have-a-conjugate-prior – Tim Nov 03 '16 at 11:18

1 Answers1

2

Your equations and your nice intuition happens for all exponential families.

I would call your $\phi$ the “observation count” rather than precision since precision is usually the reciprocal of the variance. (This is because I would interpret your Beta distribution as arising from a Beta-binomial model with $\phi$ Bernoulli observations.)

Your $(\phi s, \phi(1-s))$ are the natural parameters of the Beta distribution. When you combine the evidence of two distributions while assuming independence, you just take the pointwise product of their densities (and divide out any double-counted information). This is always the same as adding the natural parameters, which is what you've done.

Since $(\phi s, \phi(1-s))$ are natural parameters, another pair of natural parameters is $(\phi s, \phi)$. It's not surprising to see that $\phi$ is a natural parameter since in the Beta-binomial model, $\phi$ was the number of observations, and combining evidence should add the number of observations. In fact, every conjugate prior distribution of an exponential family has this “observation count” as a natural parameter.

As a contrast, for the normal distribution the natural parameters are $\left(\frac{\mu}{\sigma^2}, -\frac1{2\sigma^2}\right)$. You can try the same calculations as in your question to verify that taking the pointwise product of densities is tantamount to adding natural parameters.

Neil G
  • 13,633
  • 3
  • 41
  • 84
  • It was not explicitly an answer to my question, since my hope was more for a "story" that would describe the signal generating process with density as I derived. But your answer forced me to read and re-think the matter, giving me much deeper understanding on how the family of distributions is connected. So thanks! – Joanna F Nov 03 '16 at 20:49