Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [laplace-approximation]

27 questions
13
votes
1 answer

How are PQL, REML, ML, Laplace, Gauss-Hermite related to each other?

While learning about the Generalized Linear Mixed Models, I often see the above terms. Sometimes it seems to me these are separate methods of estimation of (fixed? random? both?) effects, but when I read the literature, I see the terms mixed. For…
humbleasker
  • 257
  • 2
  • 8
12
votes
1 answer

Comparing Laplace Approximation and Variational Inference

Does anyone know of any references that look at the relationship between the Laplace approximation and variational inference (with normal approximating distributions)? Namely I'm looking for something like conditions on the distribution being…
aleshing
  • 1,418
  • 1
  • 8
  • 21
7
votes
0 answers

Why does Quadratic (Normal/Laplace) Approximation fail on multilevel models?

In Statistical Rethinking, 2nd Edition, section 13.1, Richard McElreath says: Why doesn’t simple quadratic approximation, using for example quap, work with multilevel models? When a prior is itself a function of parameters, there are two levels of…
January Board
  • 81
  • 2
7
votes
2 answers

Bayesian inverse modeling with non-identifiable parameters?

If I have a physical model \begin{equation} y = \frac{1}{\beta_0} (\beta_1 x_1 + \beta_2 x_2) \end{equation} and want to estimate coefficients $\beta_0$, $\beta_1$, and $\beta_2$ from given data set $(y, x_1, x_2)$, with regression I would…
hatmatrix
  • 739
  • 5
  • 15
7
votes
1 answer

Reference for generalized linear mixed models using Laplace approximation

I'm fitting a generalized linear mixed model in R using the Laplace approximation. I'm looking for a reference for the Laplace approximation used for that, or a reference regarding the comparison between the PQL and Laplace methods for fitting…
Tatiana
  • 71
  • 2
7
votes
2 answers

Marginalization of GP regression hyperparameters with Laplace approximation

I am using Gaussian Processes (GP) for regression (via the gpml package for MATLAB). So far, I was optimizing the hyper-parameters by maximizing the log likelihood, but I would like to try a more Bayesian approach by (approximately) marginalizing…
lacerbi
  • 4,816
  • 16
  • 44
6
votes
1 answer

What's the relationship between Laplace approximation and Variational Bayes methods?

To be precise, I'm checking this presentation https://kaybrodersen.github.io/talks/Brodersen_2013_03_22.pdf, but I don't understand what is the connection between Laplace method and variational bayes? I mean, I know perfectly what Laplace…
Renzo Mauricio Guzmán Anaya
  • 81
  • 1
5
votes
1 answer

Constant of Laplace approximation

I'm reading Example 3.16 of Robert & Casella's Monte Carlo Statistical Methods. It uses a Laplace approximation for approximating an integral related with the Gamma distribution namely…
ZHU
  • 505
  • 3
  • 11
4
votes
0 answers

How to derive the Fisher information in the laplace approximation of a generalized linear mixed model?

I am currently using the Laplace approximation to fit some geostatistical models for binomial data. Regarding parameters estimation I do not have any problem. I can easily implement the Laplace approximation and get my estimates. But when it comes…
Emanuele Giorgi
  • 41
  • 1
3
votes
2 answers

Laplace approximation in high-dimensions

Obviously computing the inverse Hessian is hard when a probability distribution is fitted on high-dimensional datapoints. One idea to reduce computational cost would be to approximate the distribution with a diagonal Gaussian. Are there any standard…
Dionysis M
  • 794
  • 6
  • 17
3
votes
2 answers

Lower-bound on covariance estimated via Laplace approximation?

I think when a posterior is approximated to be multivariate normal as in Laplace approximation, the covariance matrix is taken to be the negative inverse Hessian evaluated at the log-posterior maximum, i.e. that $$ \mathbf{\Sigma} =…
CBowman
  • 563
  • 2
  • 13
3
votes
0 answers

Which optimizer use for laplace approximation

I have been trying to estimate the marginal posterior for D variable using Laplace approximation: $p(\theta_i) \approx \left[\frac{\det{H}}{2\pi\det{H(\theta_i)}}\right]^{1/2} \exp\left[-L(\theta_i, \psi_i^*) + L(\theta^*)\right]$ which can be…
Charlotte
  • 538
  • 2
  • 8
3
votes
0 answers

Simple approximation of joint posterior

Consider the (hierarchical) Bayesian inference problem with two unknowns $(x,\theta)$ and data $y$. I'm using a very simple ("independence"?) approximation $$ p(x,\theta|y) \approx p(x|\theta_\star,y) \, p(\theta|y) \, ,$$ where $\theta_\star$ is…
Patrick
  • 761
  • 4
  • 14
3
votes
0 answers

Is BIC useful for a multimodal distribution?

Given a dataset $D$ and a model $M$ with parameters $\theta$, the Bayesian Information Criterion can be used to approximate the model's marginal likelihood $\int p(D|\theta,M)p(\theta|M) d\,\theta$. The BIC has the form: $$BIC = 2\log…
ostrichgroomer
  • 143
  • 3
3
votes
0 answers

Laplace approximation for binomial distribution in matlab

i using bionrnd() function to generate a random vector and Laplace approximation formula to approximate the binomial distribution. but Laplace histogram dose not like the binomial distribution histogram. where is my mistake in coding? help me…
Hector
  • 131
  • 3
1
2