7

I am working with a Polya urn made of red ($r$) and white ($w$) balls. For each extracted ball, I put it back in the urn together with $c=1$ balls of the same color.

I have computed the following joint probability law of $n$ variable $Y_i$ for $i=1,2,...,n$, a variable equal to $1$ if the extracted ball is red and equal to $0$ if the extracted ball is white:

\begin{gather} P(Y_1=1,Y_2=1,...Y_k=1,Y_{k+1}=0,...,Y_n=0) = \\ =(\frac{r}{r+w})(\frac{r+1}{r+w+1})...(\frac{r+k-1}{r+w+k-1})(\frac{w}{r+w+k})...(\frac{w+n-k+1}{r+w+n-1})=\\ =\frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\frac{\Gamma(r+k)\Gamma(w+n-k)}{\Gamma(r+w+n)} \end{gather}

Since the draws from a Polya urn are exchangeable, I could alternatively have derived the same result using De Finetti's representation theorem, i.e. there exists a prior distribution $H(\theta)$ such that \begin{gather} P(Y_1=1,Y_2=1,...Y_k=1,Y_{k+1}=0,...,Y_n=0) = \\ =\int^1_0\theta\,^k(1-\theta)^{n-k}dH(\theta) \end{gather}

where $k=\sum^n_{i=1}X_i$ and $H(\theta)=lim_{n\rightarrow\infty}P\{\frac{1}{n}\sum^n_{n=1}X_i\leq\theta\}$.

My doubt is about how to find the prior distribution $H(\theta)$ such that \begin{gather} \int^1_0\theta\,^k(1-\theta)^{n-k}dH(\theta)=\frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\frac{\Gamma(r+k)\Gamma(w+n-k)}{\Gamma(r+w+n)} \end{gather}

I've read on my lecture notes that there is a theorem that guarantees the existence of a unique prior that assures this equality and, in this case, it is a Beta distribution with parameters $r$ and $w$.

Question: why is the right prior distribution a $Beta(r,w)$? How this theorem works? Are there other ways to reach the same result, other than this un-named theorem?

EDIT: I've seen my professor today and I've found out that the theorem (or the result) is by a certain Hausdorff. I've Googled it but I had no success, I thought it may be a theorem developed in another context and applied in Bayesian statistics

PhDing
  • 2,470
  • 6
  • 32
  • 57

1 Answers1

4

De Finetti's theorem proves the existence of a prior distribution $H(\theta)$ such that $$ \int^1_0\theta\,^k(1-\theta)^{n-k}dH(\theta)=\frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\frac{\Gamma(r+k)\Gamma(w+n-k)}{\Gamma(r+w+n)} $$ Looking more closely at the proof of this theorem, it seems that the prior distribution is obtained as the solution to a Hausdorff moment problem (see these notes), so the distribution is uniquely determined by its moments. So, if we can find a named distribution whose moments match those required for $H(\theta)$ then we have found $H(\theta)$.

Following the example at the end of the referenced notes, we can write $$\frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\frac{\Gamma(r+k)\Gamma(w+n-k)}{\Gamma(r+w+n)}=\int_0^1 \frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\theta^{r+k-1}(1-\theta)^{w+n-k-1} d\theta $$ From this we now see that the $Beta(r,w)$ distribution has the same moments as $H$, and hence $H$ must be the $Beta(r,w)$ distribution.

S. Catterall
  • 3,672
  • 1
  • 10
  • 18