1

I have 30 classification models. I am checking whether their mean error rates are equal(null hypothesis H0) or whether there is some model which is different(atleast one mean isnt equal, H1).

My significance level is 0.05. Do i need to account for the fact that i have 30 models and correct it to 0.5/30 although I am only testing 1 hypothesis?

I have done 5 fold cross validation and fed the data to stats.f_oneway function in python

duckvader
  • 31
  • 5
  • See http://stats.stackexchange.com/questions/88065. – whuber Dec 02 '16 at 06:10
  • Yes, while that post makes sense; with respect to ANOVA do we need to use it as ANOVA considers multiple models but there is only one hypothesis in my case? – duckvader Dec 02 '16 at 16:07
  • If you truly are testing that one hypothesis, it sounds like you are using ANOVA in its default mode exactly as intended. There isn't any multiple testing to be corrected for. – whuber Dec 02 '16 at 16:09
  • Thanks. I was confused as to whether i should consider multiple hypothesis correction due to multiple models, which clearly isnt needed now. – duckvader Dec 02 '16 at 16:18

0 Answers0