10

I am running a GAM-based regression using the R package gamlss and assuming a zero-inflated beta distribution of the data. I have only a single explanatory variable in my model, so it's basically: mymodel = gamlss(response ~ input, family=BEZI).

The algorithm gives me the coefficient $k$ for the impact of the explanatory variable into the mean ($\mu$) and the associated p-value for $k(\text{input})=0$, something like:

Mu link function:  logit                                               
Mu Coefficients:                                                      
              Estimate  Std. Error  t value   Pr(>|t|)                  
(Intercept)  -2.58051     0.03766  -68.521  0.000e+00                  
input        -0.09134     0.01683   -5.428  6.118e-08

As you can see in the above example, the hypothesis of $k(\text{input})=0$ is rejected with high confidence.

I then run the null model: null = gamlss(response ~ 1, family=BEZI) and compare the likelihoods using a likelihood-ratio test:

p=1-pchisq(-2*(logLik(null)[1]-logLik(mymodel)[1]), df(mymodel)-df(null)).

In a number of cases, I get $p>0.05$ even when the coefficients at input are reported to be highly significant (as above). I find this quite unusual -- at least it never happened in my experience with linear or logistic regression (in fact, this also never happened when I was using zero-adjusted gamma with gamlss).

My question is: can I still trust the dependence between response and input when this is the case?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
a11msp
  • 743
  • 6
  • 20

1 Answers1

1

I see no immediate reason why this should be related to GAM. The fact is that you are using two tests for the same thing. Since there is no absolute certainty in statistics, it is very well possible to have one give a significant result and the other not.

Perhaps one of the two tests is simply more powerful (but then maybe relies on some more assumptions), or maybe the single significant one is your one-in-twenty type I error.

A good example is tests for whether samples come from the same distribution: you have very parametric tests for that (the T-test is one that can be used for this: if the means are different, so should the distributions), and also nonparametric ones: it could happen that the parametric one gives a significant result and the nonparametric one doesn't. This could be because the assumptions of the parametric test are false, because the data is simply extraordinary (type I), or because the sample size is not sufficient for the nonparametric test to pick up the difference, or, finally, because the aspect of what you really want to test (different distributions) that is checked by the different tests is just different (different means <-> chance of being "higher than").

If one test result shows significant results, and the other is only slightly non-significant, I wouldn't worry too much.

Nick Sabbe
  • 12,119
  • 2
  • 35
  • 43