2

I have paired data.The response variable is categorical with levels 1-5. And I have used both McNemar's test and Wilcoxon rank sign test and I get different p-values. For McNemar's test a p-value of 0.0396; and for Wilcoxon rank sign test a p-value of 0.538. 

before=c(4,3, 5, 3, 5, 4, 3, 3, 3, 5, 5, 4, 5, 4, 5, 5, 5, 5, 5, 5, 5, 5)
after=c(4, 4, 5, 5, 4, 3, NA, 5, 3, 4, 5, 3, 4, 4, 5, 5, 5, 4, 4, 5, 4, 5)
Mik meadow
  • 203
  • 2
  • 11
  • What do these 2x22 numbers represent? Are they counts or ratings? Explain your data and your task. – ttnphns Jul 28 '16 at 11:58
  • @ttnphns re edited – Mik meadow Jul 28 '16 at 12:01
  • 1
    What question are you asking of these data? Those two tests are very different. Here is a good explanation of the McNemar Test: http://stats.stackexchange.com/questions/76875/what-is-the-difference-between-mcnemars-test-and-the-chi-squared-test-and-how – Ashe Jul 28 '16 at 14:35

2 Answers2

2

Wilcoxon's signed rank test checks if the values after are systematically higher or lower compared to those before, while the chi-squared symmetry test (aka McNemar's test in the binary case) checks for any difference in distribution, not just a shift.

So, if the true distributions before and after would differ mainly in a shift, then Wilcoxon's signed-rank test would have higher power to detect this difference in a random sample. Otherwise (e.g. if there is a tendency for values before of 3 to become 5 and vice versa), then the symmetry test would have higher power.

The problem here is: you should choose the right test before looking at the p values, simply based on the research question and the type of data you got. So it would be a bit funny to write now:

We were interested in any distributional change, so we performed a chi-squared test of symmetry and reached a significant result at the 5% level.

but actually mean:

I want to provide my readers a significant result, thus we have chosen the chi-squared test".

Michael M
  • 10,553
  • 5
  • 27
  • 43
  • 1
    (+1) Particularly for the advice to decide these things before running tests and elucidating how the tests look for differences in the paired scores. – Ashe Jul 28 '16 at 15:07
2

McNemar's is for nominal variables so it's not a proper test in this case. Wilcoxon signed rank is more appropriate.

Penguin_Knight
  • 11,078
  • 29
  • 48
  • 3
    Well, technically you are correct but http://stats.stackexchange.com/questions/65479/mcnemar-bowker-test-for-comparison-of-two-measures describes the generalisation to three or more categories which some people call Mcnemar-Bowker and other people call Mcnemar – mdewey Jul 28 '16 at 13:59
  • 1
    @mdewey, thanks. Even with the generalized version it's not appropriate. Both versions of McNemar's examined asymmetry or discordant and jumping from category 1 to 2 or from category 1 to 3 are not considered a different kind of change. Yet, the OP's data are rank data and in fact jumps from rank 1 to 2 and rank 1 to 3 are of different magnitudes and such important information should be included in the test. I've taken out the binary part from the answer to make it better. – Penguin_Knight Jul 28 '16 at 14:22
  • 2
    How do you know this is the appropriate use? The OP hasn't at all specified what they are testing for. The Wilcoxon signed-rank test and the McNemar test are extremely different in what is tested. – Ashe Jul 28 '16 at 14:34
  • 1
    @Ashe, I do not know but that they test different things has no bearing in my answer. By the fact that Wilcoxon was used I guessed it's rank data, by the fact that it's rank data I speculated that McNemar's is not a good test. It's an interactive platform and if the OP has question/clarification he/she is welcomed to put that on here. – Penguin_Knight Jul 28 '16 at 17:35