3

I have a set of integer that I am trying to see use different methods to categorize them into four groups, and the 2x2 table for the outcome of the 2 methods is displayed as below:

                method_B
     method_A   0   1   2   3
            0 182  11   0   0
            1  41 127   2   0
            2   0  12  18   0
            3   0   0   0   4

I am thinking of using chisq.test for comparing whether the difference in grouping distribution produced by 2 methods are likely to be caused by chance, but I am worrying about too many zeroes in the 2x2 table.

Should I use chi.sq test? Or is there any equivalent fisher.test for my case?

P.S. I tried fisher.test but it gives me FEXACT ERROR 501, which I have no idea what it is.

lokheart
  • 2,999
  • 7
  • 37
  • 47
  • Try increasing the `workspace` argument of `fisher.test`. That may avoid the error message, if your computer has enough memory (and you have enough patience). See `?fisher.test`. See my previous answer http://stats.stackexchange.com/questions/4023/chi-square-test-for-equality-of-distributions-how-many-zeroes-does-it-tolerate/4029#4029 – onestop Feb 14 '11 at 08:27

1 Answers1

2

I don't think either chi-squared test or Fisher's exact test is useful here. It's pretty obvious that the results from method A and from method B aren't independent of each other. A statistic to quantify how well the two methods agree is more useful. The obvious choice is Cohen's kappa.

onestop
  • 16,816
  • 2
  • 53
  • 83