How to get log-likelihood from squared deviance in Scikit Learn

Question

The score() function computes D^2, the percentage of deviance explained, but I'd like to get the log-likelihood to calculate BIC. What's the formula to go from deviance to log-likelihood?

Score function reference:

https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.GammaRegressor.html#sklearn.linear_model.GammaRegressor.score

Answer 1

If you have deviance, refer to this answer, which I'll quote below:

$$\begin{matrix} \text{Null Deviance} \quad \quad \text{ } \text{ } & & \text{ } D_{TOT} = 2(\hat{\ell}_{S} - \hat{\ell}_0), \\[6pt] \text{Explained Deviance} & & D_{REG} = 2(\hat{\ell}_{p} - \hat{\ell}_0), \\[6pt] \text{Residual Deviance}^\dagger \text{ } & & \text{ } D_{RES} = 2(\hat{\ell}_{S} - \hat{\ell}_{p}). \\[6pt] \end{matrix}$$ In these expressions the value $\hat{\ell}_S$ is the maximised log-likelihood under a saturated model (one parameter per data point), $\hat{\ell}_0$ is the maximised log-likelihood under a null model (intercept only), and $\hat{\ell}_{p}$ is the maximised log-likelihood under the model (intercept term and $p$ coefficients).

So, starting from Explained Deviance, $D_{REG}$:

$$D_{REG} = 2(\hat{\ell}_{p} - \hat{\ell}_0)$$

Therefore:

$$\hat{\ell}_{p}=\frac{D_{REG}}{2}+\hat{\ell}_0$$

You'll have to estimate $\hat{\ell}_0$ if you want to compute the exact value. If you simply want to compare models, then that term is constant among them, and can be safely ignored.

---

_{Ben (https://stats.stackexchange.com/users/173082/ben), Is R-squared truly an invalid metric for non-linear models?, URL (version: 2018-07-31): https://stats.stackexchange.com/q/359997}