3

If I estimate a collection of models predicting $Y$ by $\hat{Y}$, which methods are out there to combine these forecasts? Which methods work well/best (and why?) to improve prediction accuracy? My interest is of theoretical nature, for frequentist and bayesian approaches alike.

I am aware that this question is very open, but I want to gain an overview. Consequently, references to further sources or survey papers, book chapters, ... are also highly appreciated!

Edit: Bagging, boosting and stacking in machine learning was poposed as an answer to this question. I was very thankful for the link, and I recommend everyone interested in this question to read the post and its answers if they haven't done so already. However, the post does not answer my question: I am interested in specific methods for model averaging and bagging. The aforementioned post elaborates on the differences between the concepts of 'boosting', 'bagging', and 'stacking' rather than giving explicit different implementations. (e.g., model averaging use weights for each model. What I want to enquire with this post is the ways in which these weights can be obtained.)

Community
  • 1
Jeremias K
  • 1,471
  • 8
  • 19
  • 1
    As you have already mentioned in your title I would start with looking at methods for bagging but also check out product of expert models and the Bayesian committee machine. – RustyStatistician Feb 06 '16 at 18:53
  • Thanks! Do you know a good source for me to read up details? :) – Jeremias K Feb 06 '16 at 18:57
  • 1
    Possible duplicate of [Bagging, boosting and stacking in machine learning](http://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning) – Tim Feb 06 '16 at 18:59
  • Unfortunately no I don't. I was interested in this same question but with application to fitting Gaussian process regression models and those were the types of models that came up so I only know of them in that context but I imagine they are extendable to a richer class of models. – RustyStatistician Feb 06 '16 at 19:02
  • Hey @Tim, awesome! Thank you, just saw that it's also on the right hand side under 'Related'. I integrated your suggestion in my post - thanks for making me aware, it is a really neat explanation of the differences between the concepts :) – Jeremias K Feb 06 '16 at 21:27
  • 1
    There's an overview paper on model averaging in the Journal of Economic Surveys, see [here](http://onlinelibrary.wiley.com/doi/10.1111/joes.12044/full). It be of interest to you to check out [Clements and Hendry (1999)](https://mitpress.mit.edu/books/forecasting-non-stationary-economic-time-series) who present a theory of forecasting non-stationary time series. Their taxonomy of forecast errors will help you on the issue of improving, or at least understanding, forecast performance. As far as I know, they've worked quite a bit on developing the theory of forecasting. See recent papers also. – Graeme Walsh Feb 06 '16 at 21:36
  • 1
    Oh snap, that's a sweet overview! Have to look into it in more detail tomorrow. Hendry is always a good read, too - thank you! – Jeremias K Feb 06 '16 at 21:50

0 Answers0