comparing means of say four groups with unequal variances

Question

If I am comparing means of say four groups with unequal variances, I will have to go pair wise. So it would be 4C2 combinations.Now I can get different pair with mean significantly different, Is there any way we can claim that this specific group's mean is significantly different than all others? Suppose the groups are G1,G2,G3,G4.

Diff of G1,G2 : Significant

Diff of G2,G3: Significant

Diff of G1,G4: not Significant

Diff of G1,,G3: not Significant

.................................

....all other comb insignificant... ................... ...................

Is there any way to reach at conclusion saying mean of G3 is significantly different from all others and G3 is the main culprit?

Answer 1

You can use Welch t-tests for the pairwise comparisons, and something like Bonferoni's correction or Tukey's HSD to correct for multiple comparisons (alternatively, you could not correct it --- the choice would depend on if you're more concerned about Type 1 error or Type 2 error). If this is an exploratory analysis --- i.e. you don't have any particular hypothesis about which groups should differ --- then you should test all pairwise comparisons (using something like Tukey's HSD) and be sure to get descriptive statistics on the groups (mean and SD, probably also min, max, and anything else that might be relevant) so you have some context for interpreting the pattern of results.

If you want to directly test the hypothesis that G3 differs from the other groups and they do not differ from each other, then planned contrasts, such as Helmert contrasts are a more elegant solution. In this case, you would test the following three contrasts (or something like it):

G3 vs. mean(G1, G2, G4)
G4 vs. mean(G1, G2)
G2 vs. G1

If you were to see a significant difference for the first contrast but not the second two, you might conclude that G3 is the "culprit" (although this kind of interpretation is common, note the danger of interpreting a non-sig difference as "no difference", though, especially in underpowered designs!).

Note that in the specific example you give, though, I personally don't see evidence that G3 is different from all of the rest. G1 and G2 are significantly different, and G3 does not differ from G1. If that's your hypothesis, it looks like it's not cleanly supported by the data. If you haven't already, I recommend using a boxplot to visualize the differences among the groups, since it appears your expectations don't quite line up with the data.

Answer 2

If you don't mind relaxing your null from mean difference to stochastic size, the Kruskal-Wallis test is a nonparametric analog to the oneway ANOVA, which does not require the assumption of equal variances (nor does it require assumptions about the distributions beyond finite and i.i.d.). As the rank sum test is to the unpaired t test, so the Kruskal-Wallis is to the oneway ANOVA.

The null hypothesis of the Kruskal-Wallis test is:

$H_{0}: P(X_{i} > X_{j}) = 0.5$ for all $i \ne j$, where $i$ and $j$ identify your four groups), and
$H_{0}: P(X_{i} > X_{j}) \ne 0.5$

Post hoc pairwise tests following a rejection of the Kruskal-Wallis omnibus hull hypothesis, include Dunn's test and the less well-known Conover-Iman test.

References

Kruskal, W. H. and Wallis, A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260):583–621.

Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.

Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.