0

I'm using a sequential approach to decide the best fitting model for my data. (I'm still new to R, so I decided to go for a manual approach rather than an automated one offered by R packages).

I'm literally adding one variable at the time, trying different combination, as well as interactions.

From what I understood from previous posts on this matter, using p-values as a selective method is a sort of p-hacking and it is not suitable for inferential conclusions. but what about AIC?

I'm using AIC as selective criteria for some Negative Binomial models I'm running, choosing the models with low AIC. Can it be considered a valid modelling approach? or should I just include all the variables in the model and accept the statistical result?

Sorry if this is unclear, I can give more information if needed.

John Walk
  • 115
  • 6

1 Answers1

3

No, at least not for statistical inference: if you select your model among multiple candidates based on AIC, then the p values will not be uniformly distributed under the null hypothesis. So after such a procedure, you cannot draw any conclusions from small p values, which is the whole point of statistical inference.

See this earlier post. The comments go into how it applies to AIC, and even give at least one reference.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357