4

AIC values (from a fitted model, for example) are positive. So are the likelihood values. Are the log-likelihood values positive or negative? Here, in Wikipedia page concerning likelihood ratio test the log-likelihood values are negative and the less negative value indicates better fit.

But in this page, there is -(log-likelihood) (meaning negative of the log-likelihood) and it says that more negative value indicates better fit. In R the logLik-function from a model gives negative values, but if I have only the likelihood and I want a log-likelihood from it, I have to take negative of the logarithm???

So my question is fairly simple, when comparing those values (AIC, likelihood, logarithmic likelihood) from 2 different models, which (of which kind, more or less negative) value indicates better model?

Here is also some statistic as an example, could some one "translate" this info as words for me =) (I am using msts and tbats from forecast-function)

> fit1$likelihood
   [1] 90871.47
> fit2$likelihood
[1] 90785.92
> fit1$AIC
[1] 90909.47
> fit2$AIC
[1] 90839.92
# AIC from likelihood, par1 refers to number of fitted parameters
> 2*par1-2*log(fit1$likelihood) 
[1] -14.8344
> 2*par2-2*log(fit2$likelihood) # AIC from likelihood
[1] -10.83252

So why I wont get the same AIC values when calculating "by hand"? Which one are the correct ones?

whuber
  • 281,159
  • 54
  • 637
  • 1,101
praseodyymi
  • 67
  • 1
  • 1
  • 8
  • [AIC can be negative](http://stats.stackexchange.com/questions/84076/negative-values-for-aic-in-general-mixed-model). – Firebug Dec 09 '16 at 12:07
  • okay, I didn't know that. But still, is bigger AIC value then indicating better model or how can I compare the values? – praseodyymi Dec 09 '16 at 12:09
  • 1
    The smallest AIC is the best. – Firebug Dec 09 '16 at 12:11
  • Ah, that explains a lot – praseodyymi Dec 09 '16 at 12:12
  • 2
    I think the confusion may be due to thinking of the log-likelihood when you are really looking at the log of the likelihood ratio (likelihood for model1/likelihood for model2). So depending on which is the better model the likelihood ratio can be > 1 or < 1.hence the log-likelihood ratio can be .> 0 or < 0 respectively. – Michael R. Chernick Dec 09 '16 at 12:27
  • I think you need to explain your example in more detail. – Michael R. Chernick Dec 09 '16 at 12:44
  • Please refrain from adding more questions to your initial questions. Also, please add more details, as now you're asking about implementation details and it's completely unclear from your example where to gather this information. See my edited answer. – Firebug Dec 09 '16 at 12:45
  • You shouldn't change your question like that. Rather, create new questions. Now the answer makes no sense given the text of your question. – Firebug Dec 09 '16 at 13:15
  • @Finebug Your first comment above told the OP not to add to more questions to the initial question. So he changed the question and made things worse. He should have done what your later comment said. Got to feel bad for novices. – Michael R. Chernick Dec 09 '16 at 13:21
  • I have asked new question know. I am sorry that I dont know all the manners considering writing here (yet..). Maybe I get an answer if everyone is not already too irritated on me. – praseodyymi Dec 09 '16 at 13:25

1 Answers1

4

AIC will be negative whenever $k<\ln(L)$ (see this question and answers Negative values for AIC in General Mixed Model). Also keep in mind the log-likelihood can be positive, since the likelihood function is usually a probability density, i.e. it's always positive and can exceed $1$.

The interpretation, when comparing models, is the same though. The smallest AIC provides the "best" model, in the sense that it tends towards models with small $k$ and large $\ln{(L)}$.

Regarding likelihood, the higher the "better". Also, as log-likelihood is a monotonic transformation of likelihood, the same applies. This makes sense, since we want to minimize $\text {AIC}$:

$$\text {AIC} = 2k -2\ln(L)$$

Regarding your example in R, I'm pretty sure fit1$likelihood and fit2$likelihood are actually two times the negative log-likelihoods. The reason is simple:

fit1$likelihood - fit1$AIC
#[1] -38
fit2$likelihood - fit2$AIC
#[1] -54

If the AIC calculation is right, this means fit1 has 19 parameters and fit2 has 27 parameters.

Community
  • 1
Firebug
  • 15,262
  • 5
  • 60
  • 127