3

I have a typical problem with several variables and a large amount of data which are not important right now.

The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have performed an ANOVA so that I see which variables are able to explain the variability on $Y$. However, my question is, to what extent is this procedure (ANOVA) valid to select variables?

My selection criteria is to choose those more significant. Is it correct?

Andre Silva
  • 3,070
  • 5
  • 28
  • 55
a.desantos
  • 591
  • 1
  • 3
  • 12
  • had you perform *an* ANOVA (Y ~ X1 + X2 + ... +Xn) or *several* ANOVAs (each as Y ~ Xi)? – FairMiles Sep 12 '13 at 22:59
  • I have perform an ANOVA (Y ~ X1 + X2 + ... + Xn). BTW, what would be then the difference with ANOVAs (each as Y ~ Xi)? – a.desantos Sep 13 '13 at 06:33
  • Do you think the full model is overfitting the data much? "Several" predictors & "a large amount of data" suggests not. Cross-validate & see. If you still think you want to remove variables, certainly don't simply remove "insignificant" ones. See [here](http://stats.stackexchange.com/questions/17581/significance-testing-or-cross-validation?) – Scortchi - Reinstate Monica Sep 13 '13 at 11:18
  • Yeah, I agree, but this is not the case. My question was on feature selection based on ANOVA, is it possible to do it as I describe above? – a.desantos Sep 13 '13 at 11:52
  • What is not the case? And what are you going to do with the subset of predictors that you choose? – Scortchi - Reinstate Monica Sep 13 '13 at 12:04
  • I explain myself. what I meant by 'It is not the case' was the number of variables vs the amount of data. Let's assume that feature selection is needed because I must reduce the number of features (accelerate the procedure, simplify the model, whatever...), would be ANOVA a good method for feature reduction? In other words, the subset of predictors will be used for classification. – a.desantos Sep 13 '13 at 12:19
  • And btw, thank you for correcting me the question. I see now that I can use LaTeX syntax :) – a.desantos Sep 13 '13 at 12:23
  • No, what you describe is an arbitrary method that could lead to a model with better predictive performance but could also lead to one with worse predictive performance. Being insignificant (at what level?) is only a rough indicator that the model might be improved by leaving a predictor out; moreover, unless the predictors are orthogonal, when you remove one the significance of those remaining predictors can change. – Scortchi - Reinstate Monica Sep 13 '13 at 12:32
  • The difference is that unless X variables are orthogonal, theri "explicative power" will change when togetehre vs. when alone. If you want to maximize prediction there is no better way than leaving all of them in the model (if there were good reasons for them to be candidates) – FairMiles Sep 15 '13 at 16:29

0 Answers0