Confidence intervals for mean correlation

Question

What is the correct way to define confidence intervals for the mean of multiple correlations? I understand how to calculate CIs for individual correlation coefficients, I also understand how to calculate the mean correlation through Fisher's transformation. But what are the confidence intervals for this mean correlation value? Surely, it can not be just the +-(respective z-score of CI level) multiplied by SE of the final mean Fisher's transformed correlation score, since each single Fisher's transformed correlation score is constructed from multiple data and has it's own precision and SE.

Answer 1

Actually, what you describe is just fine.

If $\bf{x}$ $= [x_1, x_2, \dots, x_n]$, your set of data points, is approximately normally distributed, $\text{mean}(x) \pm \alpha \times\text{se}(x)$ produces a valid confidence interval, with $\alpha \approx 1.96$ for a 95% CI. As you note, since correlation coefficients aren't typically normally distributed, you can transform them using the Fisher z-transformation, calculated the mean and CI of the transformed values, and then back-transform to the original scale.

You're right that this is throwing away information: each of the coefficients has a different mean and SE, and you could obtain a better estimate by giving more weight to the more precisely-estimated correlations. This can be done, for instance, by fitting a multilevel model. However, the simple approach still yields calibrated confidence intervals.

The same thing happens when you want to calculate confidence intervals for a grand mean (a mean of means), for instance when you have multiple data points per group, and want to calculate a mean across groups.

---

Update: It's just occurred to me that you could, comments about the assumptions of your analysis notwithstanding, reframe this problem as a linear mixed model.

Let's say you currently have $n$ independent variables, $\bf x_1, x_2, \dots, x_n$, and $n$ dependant variables $\bf y_1, y_2, \dots, y_n$, and you're currently calculating $n$ correlation coefficients, $r_1, r_2, \dots, r_n$. As noted in the comments, I assume you believe that these correlations are independently sampled from a common distribution, and you're interested in calculating the mean and standard error of that distribution, for instance that you've collected $x$ and $y$ for each of the $n$ participants in an experiment, calculated $r$ for each participant, and wish to estimate $r$ at the population level.

This can be recast as a linear mixed model

$$y_i = \alpha_p + \beta_p x_i$$

where $\alpha_p$ and $\beta_p$ are the intercept and slope term for participant $p$, and $\alpha_p \sim \text{Normal}(A, \sigma_A)$, $\beta_p \sim \text{Normal}(B, \sigma_B)$. You're interested in estimating $B$, the population-level fixed effect for the slope parameter.

This can be done easily using lme4 by pivoting your data to a long format with one row per observation and columns for x, y, and participant, and using the command

lmer(y ~ 1 + x + (1 + x|participant), data=your_long_data)

Good luck!