What distribution does Fisher's exact test assume?

Question

In my work I have seen several uses of Fisher's exact test, and I was wondering how well it fits my data. Looking at several sources I understood how to calculate the statistic, but never saw a clear and formal explanation of the assumed null hypothesis.

Can someone please explain or refer me to a formal explanation of the assumed distribution? Will be grateful for an explanation in terms of the values in the contingency table.

Answer 1

In the $2\times 2$ case the distributional assumption is given by two independent binomial random variables $X_1 \sim Bin(n_1, \theta_1)$ and $X_2 \sim Bin(n_2, \theta_2)$. The null hypothesis is the equality $\theta_1=\theta_2$. But Fisher's exact test is a conditional test: it relies on the conditional distribution of $X_1$ given $X_1+X_2$. This distribution is a hypergeometric distribution with one unknown parameter: the odds ratio $\psi=\frac{\frac{\theta_1}{1-\theta_1}}{\frac{\theta_2}{1-\theta_2}}$, and then the null hypothesis is $\psi=1$.

This distribution has its Wikipedia page.

To evaluate it with R, you can simply use the formula defining the conditional probability:

p1 <- 7/27
p2 <- 14/70
x1 <- 7; n1 <- 27
x2 <- 14; n2 <- 56
# 
m <- x1+x2
dbinom(x1, n1, p1)*dbinom(x2, n2, p2)/sum(dbinom(0:m, n1, p1)*dbinom(m-(0:m), n2, p2))
[1] 0.1818838

Or use the dnoncenhypergeom function of the MCMCpack package:

psi <- p1/(1-p1)/(p2/(1-p2)) # this is the odds ratio
MCMCpack::dnoncenhypergeom(x=x1, n1, n2, x1+x2, psi)
[1] 0.1818838

Answer 2

Fisher's so-called "exact" test makes the same kind of subtle assumptions that $\chi^2$ tests make.

• The two variables being assessed for association are truly polytomous all-or-nothing variables such as dead/alive US/Europe. If one or both of the variables is a simplification of an underlying continuum, categorical data analysis should not be undertaken at all.
• There are no other relevant background variables. If $Y$ is the outcome variable and $X$ is a variable being assessed for association with $Y$, the probability that $Y=y$ is identical for every subject with $X$ fixed at $x$. Contingency tables assume in effect that there is no heterogeneity in the distribution of $Y$ that is not accounted for by $X$. For example, in a randomized clinical trial studying the effect of treatment A vs. B on the probability of death, a $2\times 2$ contengency table test assumes that every subject on treatment A has the same probability of death. [One could argue that this is too stringent an assumption, but that position doesn't recognize the loss of power from doing unadjusted tests of association.]

Fisher's test makes one assumption not made by unconditional tests of association such as Pearson's $\chi^2$ test: that we are interested in the "current" marginal distribution of both $X$ and $Y$, that is, we are conditioning on the frequencies of the $Y$ outcome categories. This is not reasonable for prospective studies. The use of Fisher's test leads to conservatism. Its $P$-values are on the average too large, because the test guarantees that the $P$-values are not too small. On the average, Pearson $\chi^2$ $P$-values are more acccurate than Fisher's, even with expected frequencies far lower than 5 in some of the cells.