In linear regression, if the random error is N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$)

Question

In linear regression, if the random error is normally distributed, does this mean the response is normal as well? In particular if $\epsilon$ ~ N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$). More specifically, I am asking if Y will have a normal distribution. I know the mean will be $\alpha + \beta X$ and variance will be $\sigma^2$ but can the distribution be assumed to be normal just because $\epsilon$ is normal? Why?

Possible duplicate of [Where does the misconception that Y must be normally distributed come from?](https://stats.stackexchange.com/questions/342759/where-does-the-misconception-that-y-must-be-normally-distributed-come-from) — Ben, Nov 07 '18 at 23:22
@Ben that link there seems more like "what are all these different sets of assumptions?" while this question seems like "why are affine transformations of normal rvs still normal?" — Taylor, Nov 07 '18 at 23:30
@s5s you can show this with moment generating functions. Have you tried that? — Taylor, Nov 07 '18 at 23:32
It's important to distinguish between the conditional distribution of Y given the value of X and the marginal distribution of Y. — Glen_b, Nov 08 '18 at 00:41

score 4 · Accepted Answer · answered Nov 08 '18 at 00:42

4

As is pointed out in this related question, the normality of the error term in a linear regression is not sufficient to ensure the marginal normality of the response variable. The latter is also affected by the distribution of the explanatory variable, which is not assumed to be normal in a regression analysis.

Under the linear regression model you have specified, the conditional distribution of $Y$ is:

$$Y|x \sim \text{N}(\alpha + \beta x, \sigma^2).$$

The marginal distribution of $Y$ is:

$$F_Y(y) \equiv \mathbb{P}(Y \leqslant y) = \int \limits_{-\infty}^\infty \Phi \Big( \frac{y - \alpha - \beta x}{\sigma} \Big) f_X(x) dx,$$

where $\Phi$ is the CDF of the standard normal distribution. In the special case where $X \sim \text{N}$ this leads to a normal distribution, but in the more general case where the explanatory variable has some other distribution, you will often get a marginal distribution for the response variable that is not normal.

answered Nov 08 '18 at 00:42

Ben

91,027
3
150
376

The problem with your answer about the conditional distribution of $Y$ is that you are assuming that the error terms have conditionally normal distributions, but this is not the usual implication. The marginal distribution of the errors is what is used to check normality of the conditional $Y$ distributions. And marginal normality of $\epsilon$ certainly *does not* imply conditional normality of the $Y$ distributions; see https://stats.stackexchange.com/a/486951/102879 – BigBendRegion Sep 13 '20 at 12:51
All regression results are conditional on $x$, so it *is* usual to assume conditional normality for the errors. – Ben Sep 13 '20 at 22:35
But that is not how the diagnostic procedure is used - it is based on the aggregate residuals, not on the residuals within each level of $X$. – BigBendRegion Sep 13 '20 at 23:04
I'm not sure I follow you. The residuals are functions of both the response and the explanatory variables, so they already account for variation in the latter. The usual diagnostic is to plot the studentised residuals against the predicted values. – Ben Sep 13 '20 at 23:30
That plot somewhat conditions on the $x$ data, but not completely with multiple regression. But that plot is not usually intended to diagnose normality. Rather, the residual histogram and *q-q* plots are the specific diagnostic tools for the regression normality assumption. These tools clearly investigate the marginal distribution of the residuals. And as the example in stats.stackexchange.com/a/486951/102879 shows, such diagnostics can show normality (and correspond to theoretical normality) when the conditional $Y$ distributions are non-normal (and even homoscedastic). – BigBendRegion Sep 14 '20 at 14:02
Thanks for the link --- I see what you're talking about now. While that is an interesting phenomenon, it does not show any deficiency in the present answer, which merely makes the usual regression assumptions (including conditional independence of the errors). (Also, it is worth noting that you can still detect the phenomenon shown in your link using standard diagnostic plots.) – Ben Sep 15 '20 at 00:49
Do you mean "independence of the errors and X" rather than "conditional independence of the errors?" The latter is irrelevant here, but the former is relevant. In any event, the conditions given by the OP do not state anything about independence of $\epsilon$ and $X$, so your conclusion "Under the linear regression model you have specified, the conditional distribution of Y is ..." is false. You might consider an edit. – BigBendRegion Sep 17 '20 at 01:37

score 2 · Answer 2 · answered Sep 13 '20 at 12:48

2

The answer is a most definitive "no." Marginal normality of $\epsilon$ does not imply that the conditional distributions of $Y$ are normal. See here for a counterexample:

https://stats.stackexchange.com/a/486951/102879

answered Sep 13 '20 at 12:48

BigBendRegion

4,593
12
22

score 1 · Answer 3 · answered Nov 08 '18 at 01:25

1

The distribution at a fixed value of x is normal. Y is not normal. Just look at the histogram of the response. It will not look like a normal distribution. But if you look at the distribution at a fixed x, then it will look normal.

answered Nov 08 '18 at 01:25

Gene

63
9

score 0 · Answer 4 · answered Nov 07 '18 at 23:46

0

Yes if $\varepsilon \sim N(0, \sigma^2)$ $Y=\alpha+\beta x+ \epsilon$ then we can say that $Y \sim N(\alpha+\beta x,\sigma^2)$. This follows from the result that if a random variable $X \sim N(\mu, \sigma^2)$ then $X+a \sim N(\mu+a, \sigma^2)$, for example if $X\sim N(0, 3^2)$ then $X+2 \sim N(2, 3^2)$

answered Nov 07 '18 at 23:46

user22546

341
1
3
10

The fact that if a random variable $X\sim N(\mu,\sigma^2)$ then $X+a∼N(\mu+a,\sigma^2)$ can be established using the method of moment generating functions. – user22546 Nov 07 '18 at 23:48
What is missing here is an implicit assumption that $\epsilon$ is normally distributed *conditional on X*. But the usual diagnostics are done unconditionally, so this is not a relevant assumption. And further, unconditional normality of the $\epsilon$ does not imply conditional normality of $Y$. See https://stats.stackexchange.com/a/486951/102879 – BigBendRegion Sep 13 '20 at 12:47

In linear regression, if the random error is N(0,$\sigma^2$) does this mean Y~N($\alpha + \beta X$, $\sigma^2$)

4 Answers4