21

What is the null model in regression and what's the relationship between the null model and the null hypothesis?

From my understanding, does it mean

  • Using "an average of the response variable" to predict the continuous response variable?
  • Using the "label distribution" in predicting discrete response variables?

If that is the case, it seems there are missing connections between the null hypothesis.

Lerner Zhang
  • 5,017
  • 1
  • 31
  • 52
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
  • 5
    Note, in R, you can try `fit = lm(formula = y ~ 1, data) ` and you should see the mean of `y`. Also, see MorganBall's answer. I would agree with his response the most. Also, a null model can be a model with $p$ predictors, with an alternative model being one with $p+k$, where k can be 1,2,... additional covariates. – Jon Feb 02 '17 at 20:26
  • 3
    Here's a reference for you: https://onlinecourses.science.psu.edu/stat501/node/295 – Jon Feb 02 '17 at 20:31

3 Answers3

21

A null model is related to a null hypothesis. Take the following univariate model:

$Y=\alpha+\beta_{1}X + \epsilon$

My null hypothesis would normally be that $\beta_{1}$ is statistically no different from zero.

$H_{0}: \beta_{1}=0$ (null hypothesis)

$H_{A}: \beta_{1}\neq 0$ (alternative hypothesis)

For a univariate linear model, such as the above, if we were to reject the alternative hypothesis then we could drop $\beta_{1}X$ from the linear model and we'd be left with

$Y = \alpha + \epsilon$

Which is your Null model and the same as mean of $Y$.

Morgan Ball
  • 1,304
  • 7
  • 20
  • 1
    To the last point, yes that is correct. In R, you can see this by comparing the intercept of `lm(y ~ 1, data)` and `mean(y)`. – Jon Feb 02 '17 at 20:29
  • 2
    +1 Nice answer Morgan! I have taken the liberty to edit your notation a tad bit, because it looked odd. – Alexis Feb 03 '17 at 00:36
17

No, I would say "null model" essentially has the same meaning as "null hypothesis": the model if the null hypothesis is true. What this means, in a particular case, of course depends upon the concrete null hypothesis.

Your interpretations as "the average value" (you probably want to say "the marginal distribution on response variable") not taking into account any predictors, is one possibility, corresponding to the null hypothesis of an "omnibus test", testing all the parameters (except the intercept) simultaneously.

But interest could well focus on a model of the form $$ y_i = \beta_0 + \beta_1^T x_{1i} + \beta_2^T x_{2i} + \epsilon_i $$ where $x_1$ contains the predictors you know are affecting the outcome, so are not wanting to test, while $x_2$ contains the predictors you are testing.

So the null hypothesis will be $\beta_2 =0$ and the null model would be $y_i = \beta_0 + \beta_1^T x_{1i} + \epsilon_i$. So it depends.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • 2
    The null hypothesis is usually something specific about parameter values; I'd say the null model would be the null hypothesis *plus all the accompanying assumptions under which the null distribution of the test statistic would be derived* -- its the assumptions that contain most of the model. For example the null hypothesis doesn't mention independence, but I'd definitely say it's part of the null model. – Glen_b Sep 17 '17 at 01:42
9

In regression as described partially in the other two answers the null model is the null hypothesis that all the regression parameters are 0. So you can interpret this as saying that under the null hypothesis there is no trend and the best estimate/predictor of a new observation is the mean which is 0 in the case of no intercept.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • 1
    This answer helped me to understand null = 0 in coefficients (other than intercept), Thanks! – Haitao Du Feb 02 '17 at 20:03
  • 1
    also, the model can be the intercept only model, compared to another model. – D_Williams Feb 02 '17 at 20:06
  • 1
    +1, this is a useful addition to the thread. However, I would say that this is a specific & very restrictive use of the term "null model". The term is often (most of the time in my guess) used more loosely. – gung - Reinstate Monica Sep 19 '17 at 15:36