Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [model-averaging]

The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.

The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.

A related although different concept is forecast combination or forecast averaging. The difference is that forecasts being combined/averaged need not be produced by models (e.g. survey forecasts).

84 questions
287
votes
8 answers

Bagging, boosting and stacking in machine learning

What's the similarities and differences between these 3 methods: Bagging, Boosting, Stacking? Which is the best one? And why? Can you give me an example for each?
Bucsa Lucian
  • 2,979
  • 3
  • 13
  • 3
40
votes
7 answers

Combining probabilities/information from different sources

Lets say I have three independent sources and each of them make predictions for the weather tomorrow. The first one says that the probability of rain tomorrow is 0, then the second one says that the probability is 1, and finally the last one says…
Biela Diela
  • 421
  • 1
  • 5
  • 5
36
votes
2 answers

Is this the state of art regression methodology?

I've been following Kaggle competitions for a long time and I come to realize that many winning strategies involve using at least one of the "big threes": bagging, boosting and stacking. For regressions, rather than focusing on building one best…
Maxareo
  • 535
  • 5
  • 11
15
votes
2 answers

Interpreting model averaging results in R

I am trying to understand and know what to report from my analysis of some data using model averaging in R. I am using the following script to analyse the effect of method of measurement over a given variable: Here is the…
Agus Camacho
  • 540
  • 2
  • 4
  • 14
13
votes
1 answer

For model-averaging a GLM, do we average the predictions on the link or response scale?

To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why? Compute the model averaged prediction on the link scale and then back-transform to the response scale, or Back transform the predictions to the…
JWalker
  • 559
  • 2
  • 8
10
votes
1 answer

Model averaging approach -- averaging coefficient estimates vs. model predictions?

I have a basic question regarding approaches to model averaging using IT criteria to weight models within a candidate set. Most sources that I have read on model averaging advocate averaging the parameter coefficient estimates based on model…
John Stella
  • 101
  • 5
9
votes
4 answers

In what applications do we prefer Model Selection over Model Averaging?

I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as Bayesian model averaging)? One example: if my…
T34driver
  • 1,608
  • 5
  • 11
7
votes
1 answer

AIC model averaging when models are correlated

AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to $$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$ where $\Delta \text{AIC}_i$ is the difference of a models AIC to the best (in terms of…
Björn
  • 21,227
  • 2
  • 26
  • 65
7
votes
1 answer

Use BIC or AIC as approximation for Bayesian Model Averaging

I want to compare "real" Bayesian Model Averaging (BMA) performed with the EM algorithm and information-criterion based BMA. Which one, BIC or AIC, is a "closer" approximation to the "real" BMA? BIC as the name is indicating? I would like to compare…
user3165675
  • 139
  • 7
6
votes
1 answer

Weights to combine different models

I have built different classification models (logistic regression, randomforest, and xgboost) for a dataset. I would like to combine the prediction of all the models to reduce the variance and increase the robustness. I read that just averaging…
Shudharsanan
  • 117
  • 2
  • 6
5
votes
0 answers

Is there justification for using cross validation scores as model averaging weights?

Bayesian model averaging uses approximate Bayes factors. Some researchers use AIC to weight models. Is there justification for using, say, the Brier score, median absolute deviation, or other such scores computed from out of sample predictions to…
Brash Equilibrium
  • 3,565
  • 1
  • 25
  • 43
5
votes
2 answers

Do "conditional" averaged coefficients *ever* make sense?

This question is related to one on Stack Overflow where the user wanted to obtain predictions and their standard errors using the "conditional" averaged coefficients from the R package MuMIn. I answered by demonstrating that if one uses vcov(model,…
Russ Lenth
  • 15,161
  • 20
  • 53
5
votes
1 answer

Are these approaches Bayesian, Frequentist or both?

currently I am comparing different combination methods: Equally Weight Averaging -> deterministic Ordinary Least Squares Averaging -> frequentist? Bayesian Information Criterion Averaging -> both, bayesian and frequentist? Bayesian Model…
Plazi
  • 247
  • 2
  • 9
5
votes
1 answer

GLMER sampling random effects

I have a model M calculated via lme4's glmer function, with random effects ("Customer ID") and fixed effects for each customer ID. My dataset is very large, so I would like to select a sample of Customer IDs, calculate the corresponding fixed and…
Alex R.
  • 13,097
  • 2
  • 25
  • 49
4
votes
1 answer

Is post-variable-selection multimodel inference a bad idea?

If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume the model is specified a priori". In this slide-show,…
Fanfoué
  • 353
  • 1
  • 13
1
2 3 4 5 6