1

I have performed a McNemar test for paired binomial variables in SPSS to see if the success rate(score 1) differs between the two conditions (the two variables: the before and after treatment variables). When I run the McNemar test using the cross-tabs function in SPSS it does not report a test-value, only the Exact Sig (2-sided), which in my case is .000, so significant. It also notes that the binomial distribution is used to determine this.

However, when I run the McNemar test using the nonparametrics tests --> related samples function, the Chi-Square is also reported, 58.061, AsympSig .000. In the end the most important thing is that there is a significant change from before to after the treatment. However, I am wondering if I should also report the Chi-Square value. The first p-value is based on the binomial distribution so I guess reporting the Chi-Square does not make sense. On the other hand, why not just report the Chi-Square and the accompanying p-value of the second method?

I hope someone has some insight on this.Thanks!

ttnphns
  • 51,648
  • 40
  • 253
  • 462
Stefan
  • 51
  • 1
  • 3
  • SPSS Statistics has McNemar test for binomial data (classic McNemar) in (1) Crosstab, (2) in "old" Nonparametrics, (3) in "new" Nonparametrics procedures. In (1), the test is exact (binomial). In (2) and (3) it is asymptotic, based on chi-square. So, results may be slightly different. You can request the exact binomial version, identical to (1), in (2) using Exact option (if Exact Module is installed on your computer). – ttnphns May 13 '17 at 16:01
  • Thank you for that explanation. Do you know when which distribution should be used? Or is it not that relevant when both versions produce a significant result, P=.000. – Stefan May 13 '17 at 16:11
  • 1
    With small sample sizes (say, less than, roughly, 70), exact, binomial is preferable. With large sample, difference will vanish. – ttnphns May 13 '17 at 16:28

1 Answers1

1

It may help you to read my answers to What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each? here, and here. The short version is that McNemar's test is actually a binomial test of whether the two off-diagonal cell counts (often denoted cells b & c) diverge from an expected null ratio of $1$ to $1$. However, it is also possible to test them as $(b-c)^2/(b+c)$. The latter quotient would be distributed as a chi-squared random variable. In other words, what SPSS seems to be reporting in your "nonparmetric" output is the test statistic for McNemar's test, not a chi-squared test of independence.

As @ttnphns notes in the comments, the two possible ways of conducting McNemar's test are asymptotically equivalent. When you sample is small, though, the binomial will be more accurate.

With regard to what you should report, that would depend on which method you used. I would report the full $2\times 2$ table, or at least the b & c cell counts, and then either the observed proportion $b/(b+c)$ with the binomial $p$-value, or the chi-squared test statistic with its corresponding $p$-value.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • gung, yes, I (being SPSS'er) think that the OP asked actually about MN chi-sq test statistic and not about Chi-square test (of homogeneity etc.) – ttnphns May 14 '17 at 11:48