3

Let's say I have an experiment which yields discrete results between 1 and $N$. I am modelling the results using a number of statistical models and want to use Akaike (corrected) or Bayesian Information Criterion to choose the best model. How can I derive AICc or BIC if the predicted variables are discrete and bounded? Is there a difference in what we mean by "sample size" in AICc or BIC formula?

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
quant_dev
  • 634
  • 4
  • 13
  • 1
    I believe AICc is derived for linear models anyway, so you'd be guessing in that case anyway. I would probably go back to the original BIC derivation and see where the $\log(n)$ term emerges from/how it would be modified for discrete/low-information responses. – Ben Bolker Sep 20 '15 at 01:24
  • In Claeskens and Hjort, "Model Selection and Model Averaging", `n` is the number of "observed data". What is "observed data"? If I am conducting a presidential election Poll and ask 1000 people "Will you vote for Bush or for Gore", than is `n=1000` (because I got 1000 binary answers) or is `n=1` (because I've got one independent data point: the fraction of the interviewees who will for vote for Bush)? – quant_dev Sep 20 '15 at 12:18
  • 1
    `n=1000`. Check [this thread](http://stats.stackexchange.com/questions/48714/prerequisites-for-aic-model-comparison) and [this thread](http://stats.stackexchange.com/questions/139201/model-selection-can-i-compare-the-aic-from-models-of-count-data-between-linear) for when it is OK to use AIC for model comparison. I mostly agree with the first thread but am suspicious of the second one. – Richard Hardy Oct 19 '15 at 18:11
  • I am shocked that there is so much ambiguity about this issues. Isn't statistics a part of mathematics? Shouldn't there be a clear proof when X works and when it doesn't? – quant_dev Oct 21 '15 at 09:01

1 Answers1

1

"I am shocked that there is so much ambiguity about this issues. Isn't statistics a part of mathematics? Shouldn't there be a clear proof when X works and when it doesn't?"

My answer: one of the most important things I ever learned was that while statistics does heavily involve math, it is not just math. It shares a lot with fields like law and politics. Some of the most important questions in statistics come down to value judgments: What's your goal? What's your data? What are you willing to assume? What can you get out of a model that is very likely to be much simpler than our incredibly complex reality.

But, yes, it would be nice to have a clear answer to the question, "When my data are discrete (in my case, binary or categorical), is the traditional AICc appropriate?"

Nicholas J. Matzke
  • 11
  • 2