2

I was calculating the deviance from a regression model that I fitted following the same idea from here Deviance.

For a likelihood $p(y|\theta)$, we define the deviance as $$D(\theta)=-2\log p(y|\theta)$$

The problem is that in this model the deviance is negative and I was trying to calculate the deviance residuals for this model, but it doesn't make much sense since $$D(\theta)=\sum (r_i^D)^2$$ where $r_i^D$ is the deviance residuals. So how the deviance residuals are calculated when the deviance is negative?

I notice from Wikipedia definition of Deviance Wikipedia that maybe this definition of deviance that I'm using is not appropriate.

  • Why are you using $\sqrt{2l_i}$? Log likelihood is always $\leq 0$ so you can't take the square root of it. – aimi May 27 '17 at 00:50
  • Correct. You're missing a minus sign inside the square root (some references use an absolute value instead). – Jacob Socolar May 27 '17 at 16:16
  • 2
    Yeah, it's not correct that the log likelihood is never > 0. If this is logistic regression, then take a look a this Q&A. https://stats.stackexchange.com/questions/166585/pearson-vs-deviance-residuals-in-logistics-regression If Poisson regression, take a look here http://web.as.uky.edu/statistics/users/pbreheny/760/S11/notes/4-19.pdf – Jacob Socolar May 27 '17 at 16:35
  • 1
    By definition, the deviance residuals depend on an observation's contribution to the log-likelihood *relative to the saturated model.* In the saturated model, by definition, the contribution to the log-likelihood is as large as it could possibly be. In any other model, that observation's contribution cannot be any greater. Therefore the *difference* attributable to the observation cannot be negative, thereby justifying (at least arithmetically) taking the square root. If you are trying to take the root of a negative value, it suggests your calculation is wrong. What exactly are you computing? – whuber May 30 '17 at 19:43
  • I wonder whether you might be using multiple, conflicting definitions of "deviance." The only relevant one here is twice the difference between the saturated log likelihood and the model log likelihood, each evaluated at their optima. That's guaranteed to be non-negative. – whuber May 30 '17 at 19:54
  • 1
    Although your calculations of the deviance residuals do not seem to be in evidence, and therefore I cannot comment on them, it appears that if you were to pursue this issue you might resolve your problem. If you find it really is just a computational error, then please flag this post before the bounty expires so we can close it and return the bounty to you. Or if you want an extended explanation of these issues--which I believe would be generally welcome--consider editing your post to help focus readers on what would be most useful to you. – whuber May 30 '17 at 20:03
  • @whuber I edited the question and deleted the comments. –  May 30 '17 at 22:00

0 Answers0