Questions tagged [waic]

WAIC stands for the widely applicable information criterion (or Watanabe-Akaike information criterion). It is used for model selection, particularly in Bayesian settings. A smaller WAIC implies that a model should have lower predictive error.

The widely applicable information criterion (or Watanabe-Akaike information criterion) is a criterion for model selection, particularly in Bayesian settings.

12 questions
12
votes
2 answers

Can WAIC be used to compare Bayesian linear regression models with different likelihoods?

I would like to use WAIC to help with model selection, where the models are simple linear regressions with Bayesian inference, non-flat priors and MCMC estimation. I am currently considering two such linear models, both of which have the same…
5
votes
1 answer

Warnings during WAIC computation: how to proceed?

I am computing the WAIC (widely applicable or Watanabe-Akaike information criterion) using the waic() function from the 'loo' package in R. When I do so, I see a warning message as follows: Warning messages: 1: 118 (5.1%) p_waic estimates greater…
Jacob Socolar
  • 1,768
  • 10
  • 18
2
votes
0 answers

WAIC for model comparisons--overly conservative?

I'm having a hard time wrapping my head around the relationship between model posterior predictions and model comparisons via WAIC. Specifically, how do I interpret findings where a model including an effect provides clear evidence for an effect,…
E.M.
  • 21
  • 3
2
votes
1 answer

DIC, WAIC in JAGS

I have a Bayesian Hierarchical model using JAGS. In order to find the best model, I have compared the DIC of two models but It's not reliable. So, I decided to calculate WAIC from JAGS. However I have a big ambiguity and i would appreciate if you…
skhalighi
  • 21
  • 2
1
vote
1 answer

Variable selection with bayesian linear mixed models (the brms package)

I am fitting a bayesian linear mixed model in R with 6 variables and 2 random effects. Inclusion of all 6 variables is motivated by a well-founded hypothesis. Does it make sense to do variable selection or can I report whether or not each fixed…
1
vote
0 answers

How to calculate WAIC from a JAGS model, and fix p_waic issue?

I am running a logistic regression type model in JAGS, and I noticed that I was getting different DIC scores (more than just a few points difference) between runs of the same model. I have a suspicion it is because I have too few 0’s in my binomial…
1
vote
2 answers

Scaling WAIC for Multiple Endogenous Response Variables

I'm trying to think about WAIC under a multivariate model scenario. Suppose I have one model composed of two relationships: y1 ~ x y2 ~ y1 This is one model. Now, I have a second model y1 ~ x y2 ~ y1 + x I would like to construct a WAIC to compare…
jebyrnes
  • 835
  • 6
  • 16
0
votes
0 answers

I'm doing model selection using DIC and WAIC. Can they be used to compare different models?

I'm comparing fitted models using the same dataset. The Models I am tweaking are Logit, Probit and a variation of Probit(Binary models). I can use DIC and WAIC to make these comparisons. Do you have a bibliographic reference that provides arguments…
0
votes
0 answers

Can you compare AIC to WAIC?

This may be a simple question, but I'm at a bit of a loss. Can I compare AIC to WAIC for the same model, one estimated using general linear models and one using Bayesian estimation? Or do I need to compare models using the same information…
0
votes
0 answers

Model comparison: WAIC value is large between nested random effect models

I have computed two models using INLA and following the proceedure laid out in this paper. As in the paper, the model has two random effects, one representing temporal relationships between data points, and one representing spatial relationships.…
SamPassmore
  • 597
  • 3
  • 18
0
votes
1 answer

Interpret WAIC value

I am trying to figure out how to interpret the WAIC value computed based on two different Bayesian models. Is the value only used for comparing the models, such that the predictive capabilities of the model with a higher WAIC value is superior? Or…
MarG
  • 115
  • 6
0
votes
2 answers

Bayes factors and predictive accuracy in model comparison in rstan / brms

Despite reading up on the subject, I can't wrap my head round it, so the question remains on shaky grounds, and responses along the lines of "read chapter x" are very welcome. What I'm doing is I'm fitting models predicting binary responses using…