1

does it make sense to assume $u\sim N(0,\sigma^2)$ when I know from a histogram that $y$ is highly skewed. Because from the assumption $u\sim N(0,\sigma^2)$ it follows that $y\sim N(x\beta,\sigma^2)$ and I'm absoluteley not sure if the assumption $u\sim N(0,\sigma^2)$ makes sense in a case where I know that the distribution of $y$ is not bell shaped. The alternative would be just to make OLS without any assumption about the error term, but in this case I can't analyze outliers and leverages (what I'd really like to do, because otherwise I can't present much more than a line which minimizes the sqaured sum of the residuals). [addendum: I can't make an outlier analysis because I can not define "outlier" in a context where I don't assume a normal distribution, because there is no outlying without a distribution] Besides your answers I'd really like to have a recommendation for a good book, where I can find some thoughts about what assumptions should we make when y is obviously not normal distributed.

Thanks for your thoughts, they'll be helpful to me!

Regards

MarkDollar
  • 5,575
  • 14
  • 44
  • 60
  • There is [Taleb's book](http://en.wikipedia.org/wiki/The_Black_Swan_%28Taleb_book%29), although it will merely point the risk of this assumption. – Wok Jul 20 '11 at 15:41
  • 1
    You need to put some more background into this. In the linear model $y_i = \alpha + \beta x_i + \epsilon_i$, it is not unusual to assume the $\epsilon_i$ are iid with a normal distribution of mean $0$, but there is no requirement for the $x_i$ or $y_i$ to be normally distributed. – Henry Jul 20 '11 at 15:41
  • Well, just because $y|x \sim N(x \beta, \sigma^2)$ doesn't mean that a histogram of the marginal distribution of $y$ will look bell shaped; I believe that will only happen if $x$ is also normally distributed. – Macro Jul 20 '11 at 16:37
  • 4
    @Henry For background you could read the *almost identical* series of questions [here](http://stats.stackexchange.com/q/11315/919), [here](http://stats.stackexchange.com/q/11351/919), and [here](http://stats.stackexchange.com/q/12262/919). ["What we've got here is failure to communicate."](http://www.youtube.com/watch?v=SnO9Jyz82Ps) – whuber Jul 20 '11 at 17:17
  • @whuber: +1 for the last link. – Nick Sabbe Jul 20 '11 at 19:08
  • @MarkDollar: you should specify in your question somewhere what 'u' is (although many seem to have correctly read it as the residual, we may all be wrong) – Nick Sabbe Jul 20 '11 at 19:08
  • 1
    @Mark Pick any $\beta$ and $\sigma$, generate 10 iid draws $\epsilon_1$, ..., $\epsilon_{10}$ from a normal(0, $\sigma$) distribution, and create the dataset $((2^i, \beta 2^i + \epsilon_i), i=1,\ldots,10)$. When $|\beta|$ and $\sigma$ are near $1$, the y's will be highly positively skewed but the data are perfectly linear with beautifully normal errors. In short, the skewness of the y's comes from the skewness of the x's but (of itself) reveals nothing at all about the distribution of the residuals. You check distributional assumptions by studying the residuals, not the y's. – whuber Jul 20 '11 at 19:44
  • @whuber: Yes you're right I was asking similar questions too often. I get your answers, but I havn't found any good literature so far and I'm not sure if I should refer on literature in this case. – MarkDollar Jul 21 '11 at 10:00
  • @Mark A standard text on regression will cover this. [Draper & Smith](http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471170828,descCd-tableOfContents.html) (at least in the 2nd edition I'm looking at) devote an early chapter (a tenth of the book) to examining residuals. For extensive practical insight on residuals in all statistical models, Tukey's [EDA](http://www.google.com/search?tbs=bks:1&q=isbn:0201076160) is worth working through. – whuber Jul 21 '11 at 13:30

1 Answers1

6

If u is the residual from a regression of y on some other variable x, then I think this is a variant of an earlier question. The residuals of u can be Gaussian even if the distribution of y is highly skewed as it may simply be that the distribution of x is highly skewed.

Consider an example of estimating temperature (y) as a function of lattitude (x); here u represents the measurment error of the thermometer (and is Gaussian). The distribution of y values in our sample will depend on where we choose to site out weather stations. If we place them all either at the poles or the equator, then we will have a bimodal distribution. If we place them on a regular equal area grid, we will get a unimodal distribution of y values, even though the physics of climate is the same for both samples and the measurement uncertainty u is normal.

Community
  • 1
Dikran Marsupial
  • 46,962
  • 5
  • 121
  • 178