Should I give more weight to goodness of fit or to conceptual approach?. Example

Question

I am running mixed effects models with percentage data.

I run my model using a gaussian distribution approach. AIC=-258, my conditional and marginal pseudo-R squares were 0.33 and 0.11 respectively (very good!). I realized that I should model it using a binomial distribution because I have percentages. Now, the results are pretty similar, but my AIC=2386 is worse and pseudo-R squares diminished a lot (0.07 conditional and 0.02 marginal).

Is this saying that the gaussian approach fits the data better and therefore I should use it preferentially? How could I justify it?

. Now, the results are pretty similar, but my AIC=2386 is worse and pseudo-R squares diminished a lot (0.07 conditional and 0.02 marginal. If AIC becomes worse, How do you justify binomial distribution and how can you say that percentage data makes a binomial data - Yes or No data. is that intuitive or irrational approach ! — , Jul 10 '18 at 05:26
Here is a somewhat similar question: https://stats.stackexchange.com/questions/142338/goodness-of-fit-and-which-model-to-choose-linear-regression-or-poisson/142353#142353 — kjetil b halvorsen, Jul 11 '18 at 13:47
I do not find any relevance of the specified quuestion or answer to the present lssue — , Jul 12 '18 at 00:07

Florian Hartig · Accepted Answer · 2018-07-11T11:50:32.300

There are a bunch of issues here

You can't compare likelihoods / deviance / AIC between models with continuous vs. count data, see, e.g. Can WAIC be used to compare Bayesian linear regression models with different likelihoods?.
Moreover, do you have discrete k/n or continuous proportions? In either case, applying an lm on the raw data is usually not a good idea (see What are the issues with using percentage outcome in linear regression?), at least use a transformation (e.g. logit is sometimes used for continuous proportions, also arcsine, but see https://www.ncbi.nlm.nih.gov/pubmed/21560670). Better use a glm, for k/n binomial, for continuous proportions it is common to use beta regression or a pseudo-binomial.

About the decision what to do - in doubt, I would simply go for the data-generating model, i.e. in case of k/n for a binomial. Check the model fit, e.g. with DHARMa, and with k/n binomial, you also have to check for overdispersion.

Thank you Florian Hartig for your answer. I have k/n proportions, but I checked for overdispersion, as you suggested and it is very high. I will have to add an observation level random effect to deal with it. — Charly, May 18 '18 at 19:15
@Charly, you should consider the community's assessment of the quality of these two answers (as expressed by the vote totals). I would urge you to be cautious about acting on Subhash's advice. You may also want to switch which answer is accepted. — gung - Reinstate Monica, May 24 '18 at 14:23

Should I give more weight to goodness of fit or to conceptual approach?. Example

1 Answers1