1

I am comparing scores of the same group of people over 7 days. I first did a Friedman's test to know where the differences were between the 7 days and I did a Wilcoxon signed rank test for the Post Hoc. Is this an appropriate test? I'm also thinking I should apply Bonferroni correction, but should I do this based on how many groups I have, or how many comparisons I make? I'm only interested in if days 2-7 are different from the first day, not if they're different from each other.

Marissa
  • 11
  • 2
  • 1
    See this post ([Friedman's test is very significant, but its post-hoc comparisons (SPSS) are not significant](https://stats.stackexchange.com/questions/175441/friedmans-test-is-very-significant-but-its-post-hoc-comparisons-spss-are-not)) for some discussion of post hocs and Friedman. It looks like some potential references may be Pereira et al (2015) "Overview of Friedman's Test and Post-hoc Analysis" Overview of Friedman's Test and Post-hoc Analysis, Communications in Statistics - Simulation and Computation, 44:10, 2636-2653. and ... – Glen_b Mar 16 '18 at 08:22
  • ... (in particular for comparisons against a single group) Garcia et al, (2010), "Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power", Information Sciences 180, 2044–2064. I have only read a little of these papers, however, so I can't really say anything about their value yet. – Glen_b Mar 16 '18 at 08:22

1 Answers1

1

The discussion cited by @glen_b has valuable information, but can can be confusing to tease out what the correct answers are.

My understanding is that Friedman test is a generalization of the sign test, so that if you were looking for a pairwise post-hoc, the sign test would be the appropriate one.

Pairwise Wilcoxon signed-rank tests would likely have more power than the sign tests, though my understanding is that the Quade test is actually the generalization of the signed-rank test. You might consider using Quade test as the omnibus and pairwise signed-rank as the post-hoc tests. It is sometimes suggested that Friedman be used if there are five or more groups, and Quade be used for four or fewer groups. But I don't have any opinion on this.

There is also a Conover test that can be used as post-hoc for Friedman's. I don't know much about this test, but in playing with the implementation in R, it seems to give reasonable results, and appears rather powerful.

~ ~ ~

For the pairwise tests mentioned above, it makes sense to use a p-value correction method, such as Bonferroni. Personally, I find Bonferroni to be too conservative for most cases. You might look in to other methods, such as those listed here.

The implementation of the Conover test may need no a-value adjustment. I'm not sure.

Sal Mangiafico
  • 7,128
  • 2
  • 10
  • 24
  • +1. I agree that Friedman [can be seen as](https://stats.stackexchange.com/a/83907/3277) a generalization of the sign test. This, however, does not mean that the backward pass, "particularization" of Friedman, yields us the sign test as the proper after-Friedman post hoc. (Neither Wilcoxon is a proper after-Friedman post hoc.) – ttnphns Mar 16 '18 at 19:06
  • (cont.) Imagine a respondent's ratings for X1 X2 X3 X4 are `4 2 7 5`. Friedman ranks it to `2 1 4 3`. Post hoc pairwise comparison of X1vsX3 shows greater distance (2 4) than in X1vsX2 (2 1). But sign test will just say that one value is greater than the other: (- +) and (+ -), respectively. – ttnphns Mar 16 '18 at 19:06
  • 1
    What do you think would be a proper after-Friedman post hoc instead of Wilcoxon? Is a Conover better then? – Marissa Mar 16 '18 at 23:34
  • Conover test is a good candidate. – Sal Mangiafico Mar 17 '18 at 00:50