11

In my dataset, I have five (ordinal) groups with an x-amount of measurement. Because homoscedasticity is violated, I performed the Friedman chi-square test to see if there are any statistical differences between the groups:

fried = stats.friedmanchisquare(*[grp for idx, grp in df.iteritems()]))

This returned a statistical difference, but now I would like to find out between which groups the differences exist. In R there is a nice solution for this (Friedman's test and post-hoc analysis, https://www.r-statistics.com/2010/02/post-hoc-analysis-for-friedmans-test-r-code/), where they use the Wilcoxon-Nemenyi-McDonald-Thompson test, but I am unable to find one for Python.

Is there a possibility to do post-hoc analyses for the Friedman test? Alternatively, what would we be a good alternative for the Friedman test that does allow me to compare between groups, e.g. a generalized estimating equation?

Community
  • 1
Robin Kramer
  • 579
  • 2
  • 5
  • 14

3 Answers3

9

I am currently looking into this issue myself; according to this paper there are a number of possibilities to perform posthoc-tests (Update: an extension regarding the use of non-parametric tests can be found here):

  • Perform the Nemenyi-test for all pairwise combinations; this is similar to the Tukey-test for ANOVA.
  • Perform the Bonferroni-Dunn-test; in this setting one compares all values to a list of control values.
  • Alternatively, one can perform step-up and step-down procedures sequentially testing hypotheses ordered by their significance. One can use Holm's step-down procedure, Hochberg's step-up procedure or Hommel's procedure.

The STAC Python library seems to include all these tests, except for Hommel's procedure.

Archie
  • 572
  • 6
  • 16
4

Complementing the other answer, since you asked about implementation of the post-hoc tests in Python: the Orange library implements the post-hoc tests (Nemenyi and Bonferroni-Dunn), including a function to draw a Critical Difference diagram [1]

http://docs.orange.biolab.si/3/data-mining-library/reference/evaluation.cd.html (see the section "CD diagram")

[1] Janez Demsar, Statistical Comparisons of Classifiers over Multiple Data Sets, 7(Jan):1–30, 2006.

Luiz Gustavo Hafemann
  • 41
  • 1
2

You can perform any of the following tests with scikit-posthocs package: Conover, Nemenyi, Siegel, and Miller post-hoc tests.

Maksim Terpilowski
  • 141
  • 3