0

I need to recreate some of the results of R in a python library, specifically LL-Null. I know that it describes

The null deviance shows how well the response is predicted by the model with nothing but an intercept.

as per Interpreting Residual and Null Deviance in GLM R

Question

Does this mean that I use an already-fit model and 0 out all the non-intercept weights? If not, how would this go? I need to implement it custom for an internal library I am building

IanQ
  • 125
  • 5

1 Answers1

2

The documentation describes a null model that is just an intercept, without any features; in other words, the null model estimates the intercept that minimizes the loss.

For instance, in an OLS model, you can show that the value $\beta_0$ that minimizes the square error $\sum_i (y_i - \beta_0)^2$ is $\beta_0=\bar{y}$ the sample mean of $y$.

It's generally the case that the sample mean of $y$ is different from the intercept of a regression model that includes one or more features/independent variables.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • Ahh, gotcha! So, you're basically saying "take the average" of the output variable? – IanQ Mar 17 '21 at 22:08
  • Not necessarily. Different models will have different losses or parameterizations, such which might imply that the optimal value of the intercept in the null model takes an alternative form. Your question references GLMs, for instance. – Sycorax Mar 17 '21 at 22:23
  • Hmm, could you elaborate on that? Say I've got GLMs all using least-squares. How would I derive what the null model for each of them? E.g a Poisson, a sigmoid, probit, etc. Do you know of any articles or resources? – IanQ Mar 17 '21 at 22:57
  • The R documentation for GLMs, which you cite, contains four references. – Sycorax Mar 17 '21 at 23:41