5

Why would the results of the ANOVA be non-significant, while a pair-wise comparison using Tukey's Wholly Significant Difference (WSD) is significant? Is their a general pattern in the means of the data that would typically produce this result?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Patrick Phoebus
  • 51
  • 1
  • 1
    While it's not exactly what you're asking, some of the discussion [here](http://stats.stackexchange.com/questions/83030/can-anova-be-significant-when-none-of-the-pairwise-t-tests-is/83083#83083), which might be helpful. – Glen_b Feb 27 '14 at 05:51
  • Thank you for responding Glen. When I compared the two extreme group means with the Tukey WSD I uncovered a significant difference when the overall distribution of all means was not significant using ANOVA. I was just wondering if that may be attributable to some particular distribution of the sample means. – Patrick Phoebus Feb 27 '14 at 06:02
  • Yes, that was understood. I am saying that the discussion relating to the *particular configurations in sample means in the answer I point to* may be relevant to this question. In particular I think the last example is likely to be especially relevant to producing the opposite case of what you seek, so consider what happens when instead of two sets of mean all at one extreme or the other, consider if you have everything in the middle, apart from two groups just far enough away to make that pair significant for the Tukey WSD. Sorry I wasn't clear before. – Glen_b Feb 27 '14 at 06:03
  • (ctd) i.e. consider this layout of group means: $1 --- (2,3,...,k-1) --- k \quad$ -- where the first and last group are just far enough separated to make the WSD significant, while (probably) keeping the ANOVA not significant. That would be the first thing I'd try. – Glen_b Feb 27 '14 at 06:10
  • Thanks Glen. That layout of group means produced a non- significant ANOVA and a significant Tukey result for the extreme means. – Patrick Phoebus Feb 27 '14 at 18:53
  • Would you like to write an answer? I can if you don't wish to. – Glen_b Feb 27 '14 at 20:10

1 Answers1

3

While Tukey's WSD can display inflated type I error under non-sphericity, you can observe this kind of issue even when the conditions are satisfied.

An example arrangement of means that can produce the result mentioned in the question is when you have every group mean but two in the middle, and the remaining two group means just far enough away to make that pair significant for the Tukey WSD, like so:

   1
                  2
                  3
                  ⁞
                 k-1
                                 k
--+---------------+--------------+------>
            group means

where the first and last group are just far enough separated to make the WSD significant, while keeping the ANOVA not significant.

As Patrick notes in comments, this kind of approach can produce the required result - nonsignificant ANOVA, with a just significant WSD on one pair.

Glen_b
  • 257,508
  • 32
  • 553
  • 939