3

I understand that one is required to run post hoc tests after the Friedman test. For example, while the Friedman test may find a statistically significant change amongst 3 treatments, one may follow with Wilcoxon signed rank tests to find where that change is (amongst which 2 treatments).

Why not just run multiple Wilcoxon signed rank tests (and divide a/3) instead? Why the Friedman?

ttnphns
  • 51,648
  • 40
  • 253
  • 462
user6306773
  • 31
  • 1
  • 3
  • It depends on the question(s) you try to answer. There is e.g. no reason to do post hoc pairwise comparisons if you are not interested in them. Similsrly there are situations where the global test is not required. – Michael M May 08 '16 at 16:02
  • Your for granted taken intention is incorrect. Wilcoxon signed rank test is not the post hoc for Friedman neither can be the basis of post hoc for Friedman. Because Friedman is an extension of sign test, not Wilcoxon. Friedman has its own post hoc. Please search `Friedman post hoc multiple comparisons`. – ttnphns Mar 13 '17 at 06:28

1 Answers1

4

The short answer is that needless multiple pairwise testing will taint your inference. That is, if you conduct multiple pairwise comparisons, the probability of falsely rejecting the NULL of in at least one of these tests increases as the number of pairwise comparisons increases.

This is the multiple testing problem that is typically introduced in a stat class to motivate Analysis of variance. Notice that a pairwise testing procedure assumes that each test is independent of one another. This assumption cannot be true since in comparing 3 groups, the same groups are repeatedly used across tests.

Often the Bonferroni method is discussed as a means of controlling for this distortion of the type 1 error rate, where the desired p-value is multiplied by as many tests as are conducted.

By first conducting a test that is designed to compare multiple parameter estimates, the correct inference can be made (as long as the assumptions for that test hold). If this test rejects the NULL of equality of parameters, then the post hoc methods can be employed to determine which parameter(s) is(are) different.

lmo
  • 767
  • 1
  • 6
  • 16
  • Just to confirm: while Bonferroni method deals with the increase in chance of a 'fluke' (multiple tests demand a lower p) it does not deal with the fact that we are treating the 3 observations as 3 separate unrelated observations (while the Friedman treats them all as part of the same series)? – user6306773 May 09 '16 at 13:01
  • Also a follow up question if I may: when I run a Friedman test with descriptive stats (in SPSS) it shows the median change across the 3 observations. Why can't I use those 3 median observations to conclude where the change occurred and the direction of the change (instead of using post hoc tests)? Perhaps because those do not provide a p value for any of those medians? – user6306773 May 09 '16 at 13:07
  • I think that is the right intuition, although replace "fluke" with "type 1 error" (reject the NULL when it is true) and "observation" with "tests." Another way to think of this is that the Friedman test (or ANOVA) tests for equality across multiple groups. If we fail to reject the NULL, then we are done. That is, there is no evidence of a difference. If we reject the NULL, then we can continue to test which are different. We have have an idea from the raw statistics, but it is important to formally test it. – lmo May 09 '16 at 13:12