1

I have looked at the other similar questions on here, but I can't seem to understand whether power is still an issue for unequal sample sizes with the homogeneity of variance being met.

Yuen (1974) and Zimmerman (1987) seem to suggest that it's only an issue when the larger sample has the smaller variance. Is this correct? Or do I have a reduced power despite having equal variances?

I'm currently responding to reviewers and it would be good to understand this a bit better and have a reference to support my point.

zvjt034
  • 11
  • 2

1 Answers1

1

This can be addressed using a simulation on your particular dataset, e.g. subsample the larger set. Just for illustrative purposes, if you take two random sets of data, then extend one of the vectors by making it repeat itself, say, 10 times, you may get different results. Test for equality of variances still shows no significant differences, but the value of the test statistic and p-value have changed (in my trial run from p = 0.148 to p = 0.047, enough to make a 'difference'). More generally, power will change as a function of sample size (see good discussion here).

But since you have real data, you can simply show whether or not your n was sufficient using unequal n power analyses (in R pwr.t2n.test in the pwr package), that would be the most convincing response to reviewer concerns.

Community
  • 1
katya
  • 2,084
  • 8
  • 11
  • Thank you that's really helpful. I use G power and the post hoc analysis says I had very little power (.11) which was the same when I entered that the group sizes were equal (based on the same total n). The effect size is .08 so very small. I don't see how saying unequal sample sizes and low power can explain this non-significant finding when the effect size is so small. Would it be more beneficial to use my sample size in a calculation with the expected effect size? It seems obvious that my power would be low if the effect size is so small. – zvjt034 Aug 12 '15 at 09:58
  • power is a function of both the effect size and sample size, so a small difference not detected with a smaller n can be detected with larger sample; this relationship is not linear, as illustrated in the example linked above. The reviewer's concern is likely related to overall low power/n, not unequal n? – katya Aug 12 '15 at 16:44
  • The reviewer was specifically concerned with unequal samples in a t test. They didn't mention power, but I can think of no other reason the t test would be affected by unequal sample sizes. Most of what I've found online says the t test is fairly robust against unequal sample sizes. Thanks for your help. – zvjt034 Aug 13 '15 at 10:31