Why is logistic regression giving a better prediction than linear reg in XGBoost?

Question

I am building a predictive model with 7 features. My target is binary. I have tried using XGBoost in R.

bst <- xgboost(data = as.matrix(trainSet[,predictors]),
               label = trainSet[,outcomeName], max.depth=10, 
               nround=1000, objective="reg:linear", verbose=0)
pred <- predict(bst, as.matrix(testSet[,predictors]), outputmargin=TRUE)
rmse(as.numeric(testSet[,outcomeName]), as.numeric(pred))

Since my target is binary, I used logistic regression. But the prediction quality goes very bad compared to linear regression. Is it OK to use linear regression or am I overfitting? Misclassification is 23.4% with linear regression but it goes 47% with logistic.

Can you provide a [reproducible example](http://stackoverflow.com/q/5963269/1217536)? It isn't clear to me if this will be answerable as posted. — gung - Reinstate Monica, Jan 13 '17 at 14:52
as @gung mentioned. with such information, this question is impossible to answer... — Haitao Du, Jan 13 '17 at 15:08
You're evaluating your model with rmse, is this a good choice for a probabilistic model with a binary response? — Matthew Drury, Jan 13 '17 at 15:09

score 2 · Answer 1 · edited Apr 13 '17 at 12:44

I think Matthew Drury had the answer. In the last line of the code, you are using rmse, which is not 0-1 loss on your testing data. That could be one reason.

On the other hand, with the information there there is no way to answer this question. Because we do not know what is trainSet and testSet.

If you want to know if you are over fitting, you can change the number of iterations (nround) and observe the performance respect to number of iterations.

Details can be found in following link (although it is the question tile has SVM, but my answer uses boosted trees as example.)

How to know if a learning curve from SVM model suffers from bias or variance?

Why is logistic regression giving a better prediction than linear reg in XGBoost?

1 Answers1