9

Study Design: I showed participants some information about sea-level rise, focusing the information in different ways, both in terms of the time-scale and the magnitude of potential rise. Thus I had a 2 (Time: 2050 or 2100) by 2 (Magnitude: Medium or High) design. There were also two control groups who received no information, only answering the questions for my DVs.

Questions: I've always checked for normality within cells -- for the 2x2 portion of this design, it would mean looking for normality within 4 groups. However, reading some discussions here has made me second guess my methods.

First, I've read that I should be looking at the normality of the residuals. How can I check for normality of residuals (in SPSS or elsewhere)? Do I have to do this for each of the 4 groups (6 including the controls)?

I also read that normality within groups implies normality of the residuals. Is this true? (Literature references?) Again, does this mean looking at each of the 4 cells separately?

In short, what steps would you take to determine whether your (2x2) data are not violating assumptions of normality?

References are always appreciated, even if just to point me in the right direction.

landroni
  • 1,003
  • 15
  • 30
Lee
  • 397
  • 3
  • 6
  • 11

2 Answers2

6

Most statistics packages have ways of saving residuals from your model. Using GLM - UNIVARIATE in SPSS you can save residuals. This will add a variable to your data file representing the residual for each observation.

Once you have your residuals you can then examine them to see whether they are normally distributed, homoscedastic, and so on. For example, you could use a formal normality test on your residual variable or perhaps more appropriately, you could plot the residuals to check for any major departures from normality. If you want to examine homoscedasticity, you could get a plot that looked at the residuals by group.

For a basic between subjects factorial ANOVA, where homogeneity of variance holds, normality within cells means normality of residuals because your model in ANOVA is to predict group means. Thus, the residual is just the difference between group means and observed data.

Response to comments below:

  • Residuals are defined relative to your model predictions. In this case your model predictions are your cell means. It is a more generalisable way of thinking about assumption testing if you focus on plotting the residuals rather than plotting individual cell means, even if in this particular case, they are basically the same. For example, if you add a covariate (ANCOVA), residuals would be more appropriate to examine than distributions within cells.
  • For purposes of examining normality, standardised and unstandardised residuals will provide the same answer. Standardised residuals can be useful when you are trying to identify data that is poorly modelled by the data (i.e., an outlier).
  • Homogeneity of variance and homoscedasticity mean the same thing as far as I'm aware. Once again, it is common to examine this assumption by comparing the variances across groups/cells. In your case, whether you calculate variance in residuals for each cell or based on the raw data in each cell, you will get the same values. However, you can also plot residuals on the y-axis and predicted values on the x-axis. This is a more generalisable approach as it is also applicable to other situations such as where you add covariates or you are doing multiple regression.
  • A point was raised below that when you have heteroscedasticity (i.e., within cell variance varies between cells in the population) and normally distributed residuals within cells, the resulting distribution of all residuals would be non-normal. The result would be a mixture distribution of variables with mean of zero and different variances with proportions relative to cell sizes. The resulting distribution will have no zero skew, but would presumably have some amount of kurtosis. If you divide residuals by their corresponding within-cell standard deviation, then you could remove the effect heteroscedasticity; plotting the residuals that result would provide an overall test of whether residuals are normally distributed independent of any heteroscedasticity.
Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
  • Ah yes, I see how to save them. I am assuming from what you say that what it saves is the residuals by group -- that is, the differences of the sample values from the cell means, not the grand mean. Should I examine the standardized or unstandardized residuals? Although, why examine residuals if it is equivalent to examining normality within the cells? This is certainly simpler. And finally, you mention homoscedasticity. I generally check for homogeneity of variance between the cells. Is this something that also might need an examination of residuals? – Lee May 04 '12 at 01:14
  • @Lee Okay. I've edited my answer to respond to your comments. – Jeromy Anglim May 04 '12 at 01:25
  • +1, there's really a lot of good info here. One note, I'm having trouble parsing parts of your 3rd bullet point, some editing may be helpful. – gung - Reinstate Monica May 04 '12 at 02:13
  • @gung Thanks for the feedback. I gave it a little edit to try to make point 3 a little clearer. – Jeromy Anglim May 04 '12 at 02:32
  • Thanks; a lot of great info here. It will be difficult to get out of my habit of looking at normality of raw data (within cells), but I will certainly give residuals consideration for future analyses. – Lee May 04 '12 at 05:47
  • I am getting some comments & confusion [here](http://stats.stackexchange.com/questions/90028/what-is-the-minimum-viable-cell-size-for-2x2-anova/90034#comment179141_90034), based on your statement "for a basic between subjects factorial ANOVA, normality within cells means normality of residuals". This is slightly incorrect: the w/i cell dists can be normal, but if heteroscedastic, the dist of residuals won't be quite normal. Do you mind correcting this? – gung - Reinstate Monica Mar 27 '14 at 14:06
  • @gung Thanks for that. I've added a few more points and qualifications. – Jeromy Anglim Mar 28 '14 at 00:11
3

Despite many introductory textbooks stressing it, you do not need Normality. With a modest sample size and the same variance within each of the groups, i.e. homoskedasticity, ANOVA will provide accurate inference on the differences in mean response between the groups. If there is reason to suspect non-constant variance - and there may well be - then heteroskedasticity-consistent standard errors can be used.

These properties are extensions of those that are well-known for the t-test; with constant variance you can use the "plain vanilla" t-test, regardless of Normality (a result known to Fisher, way back) and with non-constant variance the unequal variance also works fine without Normality. The unequal variance version is equivalent to the Wald test that uses heteroskedasticity-consistent standard errors.

guest
  • 2,381
  • 14
  • 11
  • What makes you say that normality doesn't matter? Both ANOVA and the t-test can be quite sensitive to skewness. If the underlying distribution is asymmetric, you shouldn't use either method for small sample sizes. – MånsT May 03 '12 at 08:36
  • I'd be very happy to examine references that support this statement, particularly recent ones. However without such references I have to rely on the majority of textbooks. – Lee May 04 '12 at 01:15
  • 2
    Here's [one reference](http://rctdesign.org/TechReports/ARPHnonnormality.pdf); note that they really try to break the method, with quite extreme examples. With less extreme data it'll all still work at modest sample sizes. You could also look up McCullagh and Nelder's classic text on (generalized) linear models, where they are careful to describe linear regression through "constant variance" models, as this is the assumption that matters. For robust standard errors see intro econometrics texts; the STATA software's documentation is also a good source. – guest May 04 '12 at 02:16