McFadden's Pseudo-$R^2$ Interpretation

Question

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this pseudo R-squared?

Is it a relative comparison for nested models (e.g. a 6 variable model has a McFadden's pseudo R-squared of 0.192, whereas a 5 variable model (after removing one variable from the aforementioned 6 variable model), this 5 variable model has a pseudo R-squared of 0.131. Would we would want to keep that 6th variable in the model?) or is it an absolute quantity (e.g. a given model that has a McFadden's pseudo R-squared of 0.192 is better than any existing model with a McFadden's pseudo R-squared of 0.180 (for even non-nested models)? These are just possible ways to look at McFadden’s pseudo R-squared; however, I assume these two views are way off, thus the reason why I am asking this question here.

I have done a great deal of research on this topic, and I have yet to find the answer that I am looking for in terms of being able to interpret a McFadden's pseudo R-squared of 0.192. Any insight and/or references are greatly appreciated! Before answering this question, I am aware that this isn't the best measure to describe a logistic regression model, but I would like to have a greater understanding of this statistic regardless!

Answer 1

So I figured I'd sum up what I've learned about McFadden's pseudo $R^2$ as a proper answer.

The seminal reference that I can see for McFadden's pseudo $R^2$ is: McFadden, D. (1974) “Conditional logit analysis of qualitative choice behavior.” Pp. 105-142 in P. Zarembka (ed.), Frontiers in Econometrics. Academic Press. http://eml.berkeley.edu/~mcfadden/travel.html Figure 5.5 shows the relationship between $\rho^2$ and traditional $R^2$ measures from OLS. My interpretation is that larger values of $\rho^2$ (McFadden's pseudo $R^2$) are better than smaller ones.

The interpretation of McFadden's pseudo $R^2$ between 0.2-0.4 comes from a book chapter he contributed to: Bahvioural Travel Modelling. Edited by David Hensher and Peter Stopher. 1979. McFadden contributed Ch. 15 "Quantitative Methods for Analyzing Travel Behaviour on Individuals: Some Recent Developments". Discussion of model evaluation (in the context of multinomial logit models) begins on page 306 where he introduces $\rho^2$ (McFadden's pseudo $R^2$). McFadden states "while the $R^2$ index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the $\rho^2$ measure, for ML estimation. Those unfamiliar with $\rho^2$ should be forewarned that its values tend to be considerably lower than those of the $R^2$ index...For example, values of 0.2 to 0.4 for $\rho^2$ represent EXCELLENT fit."

So basically, $\rho^2$ can be interpreted like $R^2$, but don't expect it to be as big. And values from 0.2-0.4 indicate (in McFadden's words) excellent model fit.

Answer 2

McFadden's $R^2$ is defined as $1 - LL_{mod} / LL_0$, where $LL_{mod}$ is the log likelihood value for the fitted model and $LL_0$ is the log likelihood for the null model which includes only an intercept as predictor (so that every individual is predicted the same probability of 'success').

For a logistic regression model the log likelihood value is always negative (because the likelihood contribution from each observation is a probability between 0 and 1). If your model doesn't really predict the outcome better than the null model, $LL_{mod}$ will not be much larger than $LL_0$, and so $LL_{mod} / LL_0 \approx 1$, and McFadden's pseudo-$R^2$ is close to 0 (your model has no predictive value).

Conversely if your model was really good, those individuals with a success (1) outcome would have a fitted probability close to 1, and vice versa for those with a failure (0) outcome. In this case if you go through the likelihood calculation the likelihood contribution from each individual for your model will be close to zero, such that $LL_{mod}$ is close to zero, and McFadden's pseudo-$R^2$ squared is close to 1, indicating very good predictive ability.

As to what can be considered a good value, my personal view is that like that similar questions in statistics (e.g. what constitutes a large correlation?), is that can never be a definitive answer. Last year I wrote a blog post about McFadden's $R^2$ in logistic regression, which has some further simulation illustrations.

Answer 3

I did some more focused research on this topic, and I found that interpretations of McFadden's pseudo $R^2$ (also known as likelihood-ratio index) are not clear; however, it can range from 0 to 1, but will never reach or exceed 1 as a result of its calculation.

A rule of thumb that I found to be quite helpful is that a McFadden's pseudo $R^2$ ranging from 0.2 to 0.4 indicates very good model fit. As such, the model mentioned above with a McFadden's pseudo $R^2$ of 0.192 is likely not a terrible model, at least by this metric, but it isn't particularly strong either.

It is also important to note that McFadden's pseudo $R^2$ is best used to compare different specifications of the same model (i.e. nested models). In reference to the aforementioned example, the 6 variable model (McFadden’s pseudo $R^2$ = 0.192) fits the data better than the 5 variable model (McFadden’s pseudo $R^2$ = 0.131), which I formally tested using a log-likelihood ratio test, which indicates there is a significant difference (p < 0.001) between the two models, and thus the 6 variable model is preferred for the given data set.

Answer 4

In case anyone is still interested in finding McFadden's own word, here is the link. In a footnote, McFadden (1977, p.35) wrote that "values of .2 to .4 for [$\rho^2$] represent an excellent fit." The paper is available online.

http://cowles.yale.edu/sites/default/files/files/pub/d04/d0474.pdf