0

Consider a regression model $y = \beta_0 + \beta_1 x + \epsilon$, where $\epsilon$ is a standard normal random variable. The estimate of the intercept is $\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}$. One would therefore expect the mean value of $\hat{\beta}_0$ to be $\bar{y}$ and therefore the $t$-value of $\hat{\beta}_0$ will be \begin{equation} t_0 = \frac{\hat{\beta}_0 - \bar{y}}{\text{se}(\hat{\beta}_0)}, \end{equation} where $\text{se}(\cdot)$ denotes the standard error. However, it seems that R calculates the $t$-value assuming that $\bar{y}$ is zero. Why is that?

Amey Joshi
  • 131
  • 4
  • 3
    The null hypothesis is that the true intercept term is 0. The $t$-statistic for testing this null is $(\hat{\beta}_0 - 0)/s\{\hat{\beta}_0\}$. $\bar{y}$ is not involved here. – user3294195 Nov 26 '19 at 06:46
  • @user3294195 let us ignore the hypothesis testing part. If we consider $\hat{\beta}_0$ as a random variable in itself then shouldn't its $t$-value be computed by subtracting the mean from it? – Amey Joshi Nov 26 '19 at 08:14
  • 2
    Your assumption is fallacious. The mean of the estimate of the intercept is equal to the mean of $y$ if and only if the mean of $x$ is also zero, which is not true in general. – Nick Cox Nov 26 '19 at 09:05
  • @NickCox, let's refer to the expression for $\beta_0$ derived on the page https://stats.stackexchange.com/questions/85943/how-to-derive-the-standard-error-of-linear-regression-coefficient. It does not suggest that the mean value of $\hat{\beta}_0$ is zero. – Amey Joshi Nov 26 '19 at 11:36
  • 2
    Sure, and no one is suggesting that. The point is that the usual null hypothesis is that the intercept is zero. If that test is useless to you, ignore it or test a different null hypothesis. – Nick Cox Nov 26 '19 at 11:56
  • 1
    Re: "One would therefore expect the mean value of $\hat\beta_0$ to be $\bar{y}:$" only under special circumstances. Taking expectations (with respect to $\epsilon$) we find $$E[\hat\beta_0]=E[\bar {y} - \hat\beta_1\bar {x}]=\bar{y} - E[\hat\beta_1]\bar{x}.$$ Equating that to $\bar{y}$ is equivalent to asserting $E[\hat\beta_1]\bar{x}=0,$ which can hold if and only if $E[\hat\beta_1]=0$ or $\bar{x}=0.$ Neither is generally the case. – whuber Nov 26 '19 at 14:10
  • Thank you everyone. I think, I now understand it. "If that test is useless to you, ignore it or test a different null hypothesis" is the key point. Further, I was also wrong in deriving $E[\hat{\beta}_1] = 0$. – Amey Joshi Nov 26 '19 at 14:36

0 Answers0