0

The following problem emerges from coordinate ascent variational inference in a mixture model with Dirichlet-Multinomial components. I want to compute the expectation of the log likelihood. Since my likelihoods are Dirichlet-Multinomials, that requires taking the expectation of a bunch of log(gamma function(random variable)) terms. How does one do this?

More specifically, let $\phi_k \sim Gamma(\alpha, \beta)$, where $\phi_k$ is the $k$th element of a $K$-dimensional vector $\phi$. How do I compute the following quantities?

  1. $$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \phi_k \right) \right] $$

$$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \sum_{k=1}^K \phi_k \right) \right] $$

Rylan Schaeffer
  • 689
  • 3
  • 10
  • 1
    I don't think the first expectation has a closed form solution. Maybe Monte Carlo is a good strategy here? You can use the fact that the marginals are Beta. The second one looks easy enough though, since $\sum_{k=1}^K \phi_k=1$ is deterministic. – PedroSebe Feb 13 '22 at 22:42
  • Actually, your question just revealed to me that I was wrong to say $\phi \sim Dirichlet$. What is a reasonable distribution to place on $\phi$ if $\phi$ is the parameter for the log likelihood? – Rylan Schaeffer Feb 13 '22 at 22:51
  • Maybe $\phi_k$ should follow some Gamma distribution with hyperparameters $\alpha, \beta$? – Rylan Schaeffer Feb 13 '22 at 22:55
  • 1
    If ascent is what you are after, maybe use Jensen's inequality to lower bound both of these quantities and maximize the lower bound instead? – mhdadk Feb 13 '22 at 22:55
  • @mhdadk maximizing that lower bound is exactly where this expected value of the log likelihood arises from – Rylan Schaeffer Feb 13 '22 at 22:58
  • @rylanschaeffer why not maximize log(E(...)) directly if you already have it? is it not easier to compute than E(log(...)) In this case? – mhdadk Feb 13 '22 at 23:03

0 Answers0