In their 1999 book, Yates, Moore & McCabe specify that no more than 20% of expected counts should be less than 5 and all individual expected counts should be greater than 1.
Does this mean that, for any 2 by X contingency table that no more than 20% of the cells should be less than 5?
So, for a 2 by 5 table no more than 2 (2 cells out of 10 = 20%) should be less than 5?
I am familiar with an earlier version of the book but I have not seen the specific idscussion that you refer to. So I am not exactly sure what they are getting at but I think I have a pretty good idea. The chi square test for contingency tables is only asymptotically valid. So it requires a large sample size for the null distribution to be approximately correct so that the test would be valid and the p-value would be approximately right.
For an RxC table the problem gets worse when the row and column dimensions get large. Many years ago William Cochran had a rule of thumb suggesting that the chi square approximation would be good if all the cells were >5. I believe that the authors and others may have been doing additional research to see if they could come up with a less stringent rule. Apparently there rule is that you should have less than 20% of the cells with an expected count of 5 or less. The expected count is the number you would expect to have in the cell if the null hypothesis were true. If many cells had expected counts less than 5 this would indicate that there are a lot of sparse cells and the test would not be valid.
The additional requirement that all cells have at least 1 count in them is another requirement related to the sparseness of the cells. So to be clear let's take an example. Suppose R is 5 and C is 10. Then you have 50 cells. Everyone of those 50 cells should have at least one case in it. Since 20% is 10 cells you would compute the expected count for each cell based on the formula that assumes independence between columns. The authors are saying that they would only recommend use of the chi square approximation if there are no empty cells and the computed expect count is less than 5 in no more than 10 of the 50 cells.
Just for the record, there is an option to the chisq.test function in R that allows to simulate the p-value by randomly generate a given number of independent tables instead of deriving it from the chi-squared distribution :
chisq.test(x, y, simulate.p.value=TRUE, B=2000)
From the function help page :
If ‘simulate.p.value’ is ‘FALSE’, the p-value is computed from the asymptotic chi-squared distribution of the test statistic; continuity correction is only used in the 2-by-2 case (if ‘correct’ is ‘TRUE’, the default). Otherwise the p-value is computed for a Monte Carlo test (Hope, 1968) with ‘B’ replicates. In the contingency table case simulation is done by random sampling from the set of all contingency tables with given marginals, and works only if the marginals are strictly positive. (A C translation of the algorithm of Patefield (1981) is used.) Continuity correction is never used, and the statistic is quoted without it. Note that this is not the usual sampling situation assumed for the chi-squared test but rather that for Fisher's exact test.
That could be a way to estimate a p-value without worrying about expected counts.
+1 to both @MichaelChernick and @juba. I have heard of that rule, I believe Agresti also mentions it in his books on categorical data analysis. Note however, that both that rule, and Cochran's original rule (i.e., that all cells have expected values >5), apply to expected counts, not the observed counts. This is very slippery, and it looks a little bit like you are slipping between the two from your first to your second paragraph.
The best resource I know of for these issues is:
I list a lot of related information about chi-squared and related tests here: Contingency tables: what tests to do and when?
On the other hand, with small contingency tables with few actual counts there's a legitimate question of whether we ought to always just be using Fisher's exact test these days. (Note that this is what @juba is referring to, as you can see at the end of the quote.) There's a really good discussion on CV here (albeit one that mostly argues against using Fisher's test): Given the power of computers these days, is there ever a reason not to do a chi-squared test rather than Fisher's exact test?
