Improving a boosted regression model or change?

Question

I am looking at a data set that contains multiple predictors and a continuous response. Using dismo along with gbm I built (a terrible one?) model. Using the package sROC, I got an AUC or 0.48 - so my model is actually quite worthless. Can anyone suggest any general ways to improve a model? I can post an example but ultimately, I am just wondering how experienced modelers tweak there models, etc.

Answer 1

You should figure if you model is doing poorly because it is learning too few or too much - if it is underfitting or overfitting. Once you know that, you can tweak the relevant parameters of your model to improve it.

If it still does not give you anything useful, then either there is no information in the data, or your model is especially bad at finding the pattern. Only then should switch to another model, but only after you have found a reason why model 1 was bad and model 2 will be better.

To learn how to diagnose this kind of problem, look for the following keywords and know their meaning: Bias/Variance tradeoff, Overfitting/Underfitting, Learning Curve.

Here are some starting ressources:

• CV: Question about bias variance tradeoff
• CV: Intuitive explaination of the bias variance tradeoff
• Andrew NG MOOC: The problem of overfitting
• Andrew NG MOOC: The complete serie in the chapter "What to try next"