Fisher exact test on 4 x 3 table, with low count?

Question

I'm thinking about applying Fisher Exact Test for the following contingency table:

  Answer          Category-1  Category-2
  A-1             2           15
  A-2             0           17
  A-3             0           27
  A-4             2           15

However, I'm not sure if this would be correct for two reasons: 1. Some counts are zero. 2. Typically Fisher Exact Test is used on 2 x 2 tables, not sure if that's a restriction.

Also, I'm performing the test with the R command fisher.test

It has long ago been extended to larger tables. It is preferable to the chi square test which has accuracy problems when there are sparse cells. — Michael R. Chernick, Feb 17 '19 at 04:07
3. you should wonder about whether or not it is good to condition on the margins (was the experiment performed such that the margins are fixed?) — Sextus Empiricus, Feb 21 '19 at 19:16
Isn't your table 4 x 2? The first column just looks like a label, rather than a measurement with random variation. — Sextus Empiricus, Feb 21 '19 at 19:27

score 10 · Accepted Answer · edited Jun 11 '20 at 14:32

Having observed zeros is not an issue for a Fisher Exact test -- nor indeed is it a problem for a chi-squared test (it's not clear why you think this would be a difficulty with an exact test; if you can clarify the source of your concern, additional explanation/clarification may be possible). An entire row or column of zeros might potentially be an issue for some implementations, though it's pretty easy to deal with if it arises (e.g. by combining with an adjacent row or column -- or equivalently, by eliminating it altogether - either way I think the outcome should be the same as including it in this case).

One potential issue for you (not a problem for the test per se, but which may cause you concerns in some cases) is that entire columns or rows of low counts may in some cases lead to difficulty obtaining small p-values (the lowest attainable p-value may be greater than some common choices of significance level). This tends to be less of an issue with larger tables (I believe it doesn't pose a problem for the table in your question unless you require quite low significance levels)
The test extends in a fairly natural way to $r\times c$ tables, using the likelihood (under the null) as a criterion by which to order the tables (effectively, likelihood is the test statistic; lower likelihood = "more extreme" in relation to calculation of p-values). The Fisher Exact test on an $r\times c$ table is sometimes referred to as a Fisher-Freeman-Halton test [1].

Indeed, the help on the R function you mention makes it quite clear that it works on $r\times c$ tables, and gives several references that relate to the $r\times c$ case.

[1] Freeman, G.H., & Halton, J.H. (1951).
"Note on exact treatment of contingency, goodness-of-fit and other problems of significance."
Biometrika, 38, 141-149.

I imagine that the concern with the *observed* counts and the *exact* test relates to the original rule about *expected* counts and the *chi-squared* test (https://stats.stackexchange.com/questions/30765/for-chi-square-on-any-2-by-x-contingency-table-should-no-more-than-20-of-the-c). — Sextus Empiricus, Feb 21 '19 at 19:23

score 3 · Answer 2 · answered Feb 17 '19 at 08:12

In principle, Fisher's exact test can be used on tables of any size, with any entries. The only issue is whether it is computationally feasible, which is an issue when you are using large tables with large values. In this case the fisher.test function in R can comfortably handle the matrix you are using, and the system time for execution (on my PC) is so small it does not even register as a non-zero time.

#Input data and perform Fisher's exact test
x <- matrix(c(2, 15, 0, 17, 0, 27, 2, 15), byrow = TRUE, nrow = 4, ncol = 2);
fisher.test(x);

    Fisher's Exact Test for Count Data

data:  x
p-value = 0.1434
alternative hypothesis: two.sided

#Check system time for test
system.time(fisher.test(x));

 user  system elapsed 
    0       0       0

Fisher exact test on 4 x 3 table, with low count?

2 Answers2