Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [variational-inference]

24 questions
2
votes
2 answers

Inference: How is the Laplace approximation actually useful to us compared with MLE and MAP?

I was reading a few different sources (including the "Machine Learning and Pattern Recognition" book by Bishop) about the Laplace integral approximation method for inference. However, I am still confused about the use of this method within…
Rocky the Owl
  • 21
  • 2
1
vote
1 answer

Variational Autoencoder assumtions

I am currently reading the paper "Importance Weighted Autoencoders" and am having a hard time understanding something regarding the original Variational Autoencoder (VAE) as described here In the first paragraph of the third subsection the author…
Ofek Glick
  • 63
  • 7
1
vote
0 answers

Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation

I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it lies on the real coordinate plane. Let $\zeta =…
James
  • 11
  • 2
1
vote
1 answer

Variational Bayes for a univariate Gaussian

I'm following an example from Murphy's book (Sec 21.5.1) on how to apply Variational Bayes to infer the posterior over the parameters for a 1D Gaussian $p(\mu,\lambda|\mathcal{D})$. The example uses a prior of the form $$ p(\mu,…
Blade
  • 611
  • 4
  • 16
0
votes
0 answers

Distribution over parameters vs. distribution over functions

I find it hard to distinguish between these two concepts. In a variational inference setting we learn a distribution over the parameters of our function. in the definition of Gaussian processes we learn a distribution over functions. I wonder, when…
samsambakster
  • 157
  • 6
0
votes
0 answers

Expected value of log(gamma function(Dirichlet variable))

The following problem emerges from coordinate ascent variational inference in a mixture model with Dirichlet-Multinomial components. I want to compute the expectation of the log likelihood. Since my likelihoods are Dirichlet-Multinomials, that…
Rylan Schaeffer
  • 689
  • 3
  • 10
0
votes
0 answers

Bayesian Interpretation of Deep Ensembles

I was wondering if training a neural network in the deep ensemble setting can lead to a network with a posterior vs. a point estimate architecture? Recently there have been discussions over the interpretability of Deep ensembles as Bayesian models.…
samsambakster
  • 157
  • 6
0
votes
0 answers

Best method(s) to estimate the parameters of a stochastic process with a hybrid (i.e. switching) random input variable?

I'm looking for the best approach to and/or methods of solving the following inference problem. I have tried searching for similar questions but don't have enough knowledge on the various methods (variational inference, SVI, MCMC, HMC, MLE,…
Bill
  • 123
  • 5
0
votes
0 answers

Variational inference - posterior predictive distribution

So suppose we have a neural network that aims to map values from one distribution to another. That is to say the inputs do not belong to the same distribution as the targets. It then follows that, technically the usual formulation of the ELBO would…
user2793618
  • 169
  • 5
0
votes
0 answers

Inhowfar does Variational Inference work better with members of the exponential family?

I am reading Variational Inference: A Review for Statisticians. Working in [the exponential] family simplifies variational inference: it is easier to derive the corresponding CAVI algorithm, and it enables variational inference to scale up to…
direstraits2001
  • 23
  • 5
0
votes
0 answers

Variational inference: Rewriting ELBO

I was following some tutorials on variational inference and ELBO loss functions and I think I understand it quite well, but I'm struggling with math. If I understand it correctly we went from this…
F.Hand
  • 1
  • 2
0
votes
0 answers

Loss Function of a Variational Autoencoder when using Implicit Reparameterization Gradients (Dirichlet distributed latent space)

I would like to implement a VAE with a Dirichlet distributed latent space in Python. Since the reparametrization trick does not work for the Dirichlet Distribution I would use Implicit Reparameterization Gradients for my endeavor. Luckily, this is…
direstraits2001
  • 23
  • 5
0
votes
0 answers

Variational Autoencoder with Dirichlet distributed latent space using the Weibull Distribution

My goal is to create an VAE with an Dirichlet distributed latent space. Since the reparametrization trick does not work for the Dirichlet Distribution, I am trying to approximate the Gamma Distribution with the Weibull Distribution, from which I…
direstraits2001
  • 23
  • 5
0
votes
1 answer

Can someone explain Stein's method/discrepancy in a way that makes sense?

I have been wanting to understand this paper in a deeper way for a long time Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm But everytime I read about Stein's method or Stein's Lemma I get confused and I am not…
Joff
  • 599
  • 2
  • 13
0
votes
0 answers

Deriving variational posterior on the Ising model using mean field approach

I'm following an example from Murphy's book (Sec 21.3.2) on how to derive the mean field update equations to approximate a variational posterior on the Ising model Problem: let $x_i\in\{-1, 1\}$. We have a joint model of the form $$ p(\mathbf{x},…
Blade
  • 611
  • 4
  • 16
1
2