3

Consider the (hierarchical) Bayesian inference problem with two unknowns $(x,\theta)$ and data $y$. I'm using a very simple ("independence"?) approximation $$ p(x,\theta|y) \approx p(x|\theta_\star,y) \, p(\theta|y) \, ,$$ where $\theta_\star$ is the mode of $p(\theta|y)$ -- using other point estimates for $\theta_\star$ is also an option.

Question: does this type of approximation have a name? Note that

  • Unlike "Laplace approximation", I do not want to integrate out $\theta$.
  • Unlike "Variational Bayes" I'm not approximating $p(\theta|y)$ and $\theta_\star$ is obtained in a much simpler fashion.

I'm asking so that I may look up some literature on it. For instance, how might it compare to mean-field approximations?

Edit: I think it might be appropriate to call it "empirical Bayes without marginalization". Still, any insights/opinions would be welcome.

Patrick
  • 761
  • 4
  • 14
  • If $\theta^{\ast}$ is the mode of $p(\theta|y)$, then I think what you are doing on the right hand side of the equation is some form of type-II Maximum Likelihood. I.e., you take $\theta$ as some hyperparameter influencing both $x$ and $y$, and rather than integrating out the parameter uncertainty about $\theta$ encoded in a potential prior distribution, you use the ML estimate. That being said, I find the approximation itself a little strange, since it basically says $P(x,\theta|y) \approx P(x|\theta = \theta^{\ast}, y)$. Are you sure that is reasonable to assume? – Jeremias K Nov 27 '17 at 13:43
  • @JeremiasK so you agree with the edit: it's Empirical Bayes (without marginalization)? As for its validity, it is of course questionable, but not more so than EB, afaik. Also, you seem to be missing $p(\theta|y)$. – Patrick Nov 27 '17 at 17:41

0 Answers0