1

I’m running a multilevel model in which the outcome variable is a Fisher Z-transformed correlation coefficient. The residuals look quite non-normal:

QQ plot of non-normal residuals

In case the details matter, I’ll mention that the model in question is model ‘1’ described here

Though I'm not sure how concerning this is in absolute terms, it's far more non-normal than I'd previously seen when previously doing classical or multilevel regression. However, typically I don't use outcome variables that have been transformed in this way.

I'm wondering the following:

Was the non-normality of the residuals an inevitable (or highly likely) consequence of my choice of outcome variable. If so, why?

What, if anything, should I do about this issue? I’m weighing the pros and cons of doing nothing as against applying a further transformation.

In terms of how concerned I should be about this issue, is there any reason I should be more/less concerned seeing this happen with a multilevel regression model than I would be if I was doing a classical regression model and saw the same level of non-normality of residuals? I’m aware that, as per this question, in linear models in general the normality of residuals is “barely important at all” for the purpose of estimating the regression line.

Edit: As per @nick-cox's comment, I've pasted a few histograms below indicating the distribution of the original correlations and of the Fisher-transformed correlations. The reason all the correlations are positive is that the correlations are measures of effect size. I've also shared the data here.

Untransformed correlations

Transformed correlations

user1205901 - Reinstate Monica
  • 11,303
  • 26
  • 77
  • 152
  • 1
    Can you post plots of the distribution of the original correlations and their transforms? – Nick Cox Sep 28 '20 at 09:40
  • Thanks, I just posted histograms of the original correlations and the Fisher-transformed correlations, and also a link to the data (raw residuals, original correlations, and Fisher-transformed correlations). I could certainly post more/other plots or more data if it would be helpful. – user1205901 - Reinstate Monica Sep 28 '20 at 10:26
  • 1
    So, the transformation makes things worse. I've not wanted to use correlation as an outcome variable, but for once I would prefer -- on this information -- to use it directly. A linear model in principle might predict values for $r$ outside $[-1, 1]$ but my guess is that you won't be bitten by that in practice. – Nick Cox Sep 28 '20 at 10:41
  • Even a prediction outside $[0,1]$ would be problematic since the correlations are effect sizes and can only be positive. Originally I thought to use a Beta mixed model, as per [this question](https://stats.stackexchange.com/questions/233366/how-to-fit-a-mixed-model-with-response-variable-between-0-and-1), but in $\frac{5}{2032}$ of my cases $r=0$. Workarounds are suggested [here](https://stats.stackexchange.com/questions/48028/beta-regression-of-proportion-data-including-1-and-0) but it seems a bit complicated. Do you think a Beta mixed model would be preferable to using $r$ as the outcome? – user1205901 - Reinstate Monica Sep 28 '20 at 11:21
  • 1
    I disagree. A prediction of a negative correlation tells you about what your data are saying. Not letting the data tell you about a problem is quite different from not letting a model predict an impossibility. I don't know enough about beta mixed models to advise otherwise. – Nick Cox Sep 28 '20 at 12:32

0 Answers0