1

I am currently attempting to determine the most predictive weighted multiple linear regression model to use and am trying to figure out the best combination of variables to use in the model.

My first idea was to use take the weighted mean of errors from LOOCV for all possible variable combinations. However, being that my data has 20 variables, this would forever for my computer to run all of these regressions. After doing a bit of research, my impression is that using LOOCV for model selection is similar to comparing the AIC of each run of the regression. Is this true? If so, would another method for determining the best model be by running a single regression using all observations for each variable combination and then choosing the model with the lowest AIC? Would the results be impacted by the fact that the observations in my model are weighted differently?

Any help would be greatly appreciated!

dwm8
  • 263
  • 1
  • 3
  • 8
  • See [Algorithms for automatic model selection](http://stats.stackexchange.com/q/20836/17230) for problems with what you're suggesting. A sensible first step would be to validate your twenty-predictor model & see if it is even over-fitting. If variable selection seems like a good idea (alternatives include data reduction of the predictors or shrinkage without selection) @MikeHunter's answer gives sound advice – Scortchi - Reinstate Monica Jun 01 '15 at 19:10

1 Answers1

1

Why not make your life much simpler and use the Lasso or LARS for first-pass variable selection? Tibshirani's Lasso technique has been widely vetted, is ranked as one of the top statistical contributions of recent years on NormalDeviate.com and its approach to L1 normed regularization is solid.

Mike Hunter
  • 9,682
  • 2
  • 20
  • 43
  • Thanks for the help. I just ran my data through the `lasso()` function in MATLAB, though I am a bit unsure about how to interpret the results. Is there a rule of thumb for choosing Lambda? Which model of those provided is the 'best'? Once I have found the model that tells me which combination of coefficients are most predictive, do I use the coefficient results provided by the lasso function to use for the regression, or do I re-run a multiple linear regression using the remaining variables and use those coefficients? – dwm8 Jun 04 '15 at 14:54
  • I don't know how Lasso works in Matlab but you can play around with Lambda. And, yes, if you get a subset of regressors, I would rerun that subset in a regression module and, after checking that the results are as expected, use those coefficients, not Lasso's. – Mike Hunter Jun 04 '15 at 14:57