0

I have binomial response data with a single explanatory variable. In R I can write the model as something like

myglm = glm(success_rate ~ x, family='binomial', weights=n_attempts)

My question is whether the confidence intervals computed by confint(glm) are correct, given that I'm using weights. The documentation suggests that they should be. But if I run a permutation test, randomly switching the order of x, the coefficient of x is significant at the 5% level about 15% of the time. This seems like utter madness and a ridiculously high false discovery rate relative to the ~5% you'd expect. The same thing happens with different predictors, too.

Any input would be much appreciated.

Will
  • 1,118
  • 8
  • 16
  • this sounds like two problems to me.. the confidence interval is calculated by taking the square root of the diagonal of variance covariance matrix (see https://stats.stackexchange.com/questions/241449/matrix-and-regression-model) . You are running the glm with a rate as response so this is definitely ok – StupidWolf Jun 04 '20 at 22:27
  • the fact that permuted dataset reaches significance more than expected can be due to other issues, for example, if your data is overdispersed and you underestimate the variance. can you run the same formula with family = quasibinomial? See if it is over-dispersed – StupidWolf Jun 04 '20 at 22:28
  • Quasi binomial does indeed indicate overdispersion. Estimated dispersion parameter is about 4. I ended up including a random intercept at the subject level and the permutation test makes more sense now. – Will Jun 06 '20 at 11:23

0 Answers0