What does it mean when glm algorithm doesn't converge but still gives results

Question

I'm running many glm models in R (negative binomial regression to be specific) to a fairly large dataset (N = 175,000) with the intention of performing a specification curve analysis. For my case, this includes running simple, single-variable model specifications (e.g., glm.nb(y ~ x, data)) as well as specifications with up to 8 control variables (e.g., glm.nb(y ~ x + a + b + c + d + e + f + g + h, data)). My x and y are count variables and my controls are a mix of binary and count variables.

In the process of running these models I have occasionally been given the error, Warning message: glm.fit: algorithm did not converge. I have fixed this in the past my increasing the maximum number of iterations (e.g., glm.nb(y ~ x + a + b + c + d + e + f + g + h, data, maxit = 1000)), but I still get the error sometimes. However, the model is still producing results, which seem pretty sensible when compared to the results of models that do not produce the error message.

What does it mean when glm gives the error but still produces results? Are the results invalid?

Answer 1

I would like to use logistic regression as an example to explain what is happening when algorithm did not converge.

We know that for perfect separation case, logistic regression without regularization will not converge.

You may review here: Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

Suppose we have a perfect separation in logistic regression, the algorithm is trying hard to find a solution, that can minimize the logistic loss. However, because we can make the loss even by increase the parameter value. The algorithm will keep doing it, until it exceeds the max number that can be represented by a computer.

Therefore, the algorithm will end somewhere, in most cases, it will end with the max iteration. The ending may not be bad, i.e., the parameters still can minimize the loss to some level, this is why you will see, even the algorithm is not converge but the model is still working.

Here is an example, from similar to my previous answer, that you can see, for perfect separation, the algorithm does not converge but we are still getting a "meaningful" output

set.seed(0)  
d=mlbench::mlbench.2dnormals(100, 2, r=3)
fit = glm(d$classes~d$x, family = binomial())
plot(d)
abline(0, -fit$coefficients[2]/fit$coefficients[1], col='blue',  lwd=2)