1

I have ordinal data from 3 related samples. First i conducted a Friedman test to know if there were differences between at least two samples. Next I conducted a Wilcoxon signed rank test to know between which of the 3 samples there were differences and i generated 3 possibles pairs. However i don't know if i need to apply Bonferroni correction and basically compare my Asymp Sig (2 tailed) with P=0.05/3 instead of 0.05, or if i should mantain a p=0.05 to know if i accept or reject my null hypotesys.

Could anyone help me to clarify this question? Thank you very much.

sofia
  • 11
  • 3
  • One possibility is to consider the [Quade](https://stats.stackexchange.com/questions/211453/why-run-a-post-hoc-wilcoxon-signed-rank-test-after-friedman-test-why-not-just-s) test or the Friedman test (which is more akin to the sign test than the signed rank but is also suited to multiple related samples) – Glen_b Jun 30 '17 at 10:49
  • First I conducted a Friedman test, but that only tells me that differences exist at least between two samples. I need to conduct a Post-hoc to know between which samples there are differences, right? – sofia Jun 30 '17 at 11:09
  • 1
    You should probably clarify this in your question – Glen_b Jun 30 '17 at 11:14
  • Thanks for the input. Can you help me knowing what you know now about my analysis? – sofia Jun 30 '17 at 11:29
  • I think you'll get better answers on post-hoc testing from others. – Glen_b Jun 30 '17 at 11:52

2 Answers2

1

It depends on what kind of false discovery probability you wish to control.

Keeping p=0.05 means you allow 5% probability of false discovery (e.g. rejecting null hypothesis while it shouldn't be rejected) in each comparison.

Taking Bonferroni correction means that you allow 5% probability for at least one false discovery.

If the latter is what you deserve, I'd recommend using some other correction (eg. Holm), because Bonferroni is very conservative. It actally dumps false discovery probability far below 5%. But, on the other hand, with 3 comparisons only this effect is not very big.

Holm correction:

First, order your p-values in incereasing order. Let $p_1$ be the smallest and $p_n$ the largest one.

Then compare each $p_i$ with $$\frac{\alpha}{n-i+1}$$

So, in your case ($n=3$):

the smallest p-value ($p_1$) is significant if it's lower than 0.05/3,

the middle one ($p_2$) is significant if it's lower than 0.05/2,

and the largest ($p_3$) is significant if it's lower than 0.05.

Plus, some people say that if you find $p_i$ significant you should also consider all lower p-values significant. So according to them, if, for example, $p_3$ is significant, while $p_2$ is not, you should also treat $p_2$ as significant.

Łukasz Deryło
  • 3,735
  • 1
  • 10
  • 26
1

There are some post-hoc tests devised for Quade and Friedman tests. The best place I can refer you to is the documentation and vignette for the PMCMR package in R. The methods there are simply called "Friedman – post-hoc test after Nemenyi", "Friedman – post-hoc test after Conover", and "Quade – posthoc tests". Having played with them a little bit and here, they seem to give reasonable results, but I can't speak to their statistical qualities.

Sal Mangiafico
  • 7,128
  • 2
  • 10
  • 24