1

I am trying to run a program which generates data from various covariate distributions, and finds the maximum likelihood estimator by explicitly maximising the log likelihood function. I am specifically working with a logistic regression model. The problem is that I keep getting nan for my log likelihood values, because some probabilities are inevitably going to be sufficiently close to 0 or 1 that it gives problems in the log. This means that the optimisation method won't work!

This problem was also asked here in this post and the answer said to use the scipy.stats.norm.logcdf method or scipy.stats.norm.logsf but I cannot find any thorough documentation about these or how to use them. How and if it works to solve the problem. I would appreciate if someone could point me towards an example of using these, and explain what the output is.

Meep
  • 153
  • 6
  • The cure often depends on the specific circumstances. Please see [the many other posts about this issue](https://stats.stackexchange.com/search?q=max*+likelihood+log+infinite+is%3Aanswer) that have appeared. – whuber Sep 13 '19 at 20:17
  • @whuber I have gone through the posts that were relevant to my question. Unfortunately, most of the posts simply give an explanation of the prob;em, rather than a suggestion of what to do about it. The closest I have found regarding my specific case is this post: https://stats.stackexchange.com/questions/277201/is-it-possible-to-get-fitted-values-0-or-1-in-logistic-regression-when-the-fitti/277237#277237 but it only shows that, in R, the problem isn;t so much of one because the algorithm converges close to the true beta values anyway. That is not the case in my python script!!! – Meep Sep 14 '19 at 18:18
  • I have been investigating the scipy.stats.norm.logcdf. I think I vaguely get it, but I cannot see that it can be applied to arbitrary distributions. – Meep Sep 14 '19 at 18:20
  • One question is *why* your script runs into problems. The reasons could range from (1) bugs to (2) a search that goes beyond stipulated boundaries to (3) use of a grossly incorrect model to (4) poor choice of initial values in a search to (5) a poor algorithm to (6) inadequate protection against computing logarithms of tiny numbers. Without the details it's therefore difficult to point to a specific solution. – whuber Sep 16 '19 at 10:14
  • I always try to protect divisions and logs. The precise method is dependent on the solver and the problem. Some solvers support bounds on the decision variables. In other cases you may protect things in code. – Erwin Kalvelagen Sep 19 '19 at 05:44

0 Answers0