Can Fisher's exact test accept a 'vector of probabilities'?

Question

The R function chisq.test allows a 'vector of probabilities' to be set with the p argument. Is there an equivalent for fisher.test()?

I'm trying to test whether there's a significant difference between the proportion of people in three unequally-sized groups who studied STEM subjects at university.

I get a Chi-squared approximation may be incorrect warning, which I think is due to the small samples, so thought I'd try Fisher's.

As the three groups are unequal sizes, when using the chisq.test I'm feeding in the probability distribution. I'm not sure how to do this with fisher.test.

There's an R-Fiddle of the Chi-square implementation here.

---

My null hypothesis is that none of the three groups have a higher proportion of STEM (Science, Technology, Engineering, Maths) graduates than any other.

Group one constitutes 195 people (or 53% of the sample i.e. the 0.5313351499 in the R-Fiddle), Group two is 134 people (or 37%) and Group three is (38 people, or 10%). Group one has 22 STEM graduates, Group two has 16 and Group three has 9.

Answer 1

You appear to have confused some things together.

There's more than one chi-squared test.

For example, there's

(i) the goodness of fit test, which you would specify probabilities for;

(ii) the test of independence, which you would not specify probabilities for; it doesn't make sense in this case - the expected values follow from the null hypothesis (and the assumptions).

(iii) the test of homogeneity. This is usually performed as a chi-squared test, in which case the conditioning on both margins makes it equivalent to the test of independence.

It is the second kind of case (testing for dependence against the null of independence) that the Fisher exact test corresponds to. It doesn't make sense to specify probabilities for that case.

Your circumstance -- "test whether there's a significant difference between the proportion of people in three unequally-sized groups who studied STEM subjects at university" -- sounds like a test for homogeneity, in which case you could test it by conditioning on both margins and using the Fisher exact test. (That's what I expect Fisher would have done.)

---

Exact tests for the goodness of fit case

While the usual Fisher exact test doesn't apply to the goodness of fit case, that doesn't mean you can't construct an exact test for the goodness of fit situation. Indeed it should be straightforward.

You can specify some statistic - even one like the "probability of the table itself" that's used in the Fisher test, though for myself I'd lean toward the chi-square statistic - and then compute the exact probability of all tables more extreme (by the ordering induced by the choice of test statistic) than your observed one under the usual multinomial assumption.

Indeed, in R, chisq.test can essentially give you this (up to sampling error, which you can control), by simulation. You set the simulate.p.value argument to TRUE and choose the number of replicates (simulated tables), by setting the argument B. I usually choose a larger value than the default for B (simulations are fast). Here's an example (with the result simulated twice so you can see the slight variation in p-value):

> p=c(.1,.2,.3,.4)
> x=c(3,5,16,10)
> chisq.test(x,p=p,simulate.p.value=TRUE,B=10000)

    Chi-squared test for given probabilities with simulated p-value (based on 10000
    replicates)

data:  x
X-squared = 4.7745, df = NA, p-value = 0.1904

> chisq.test(x,p=p,simulate.p.value=TRUE,B=10000)

    Chi-squared test for given probabilities with simulated p-value (based on 10000
    replicates)

data:  x
X-squared = 4.7745, df = NA, p-value = 0.1922

There are also some R packages that might be of assistance in the goodness of fit case if you were trying to do a precise calculation of p-value (whether complete enumeration or enumeration of all more-extreme cases)

Answer 2

What you did isn't the way you run a chi-squared test. You need a contingency table. For each group, you have some people in STEM fields and some people who aren't. Thus, you will have two rows of counts, or two cells per group. Then you run a chi-squared test of the independence of the rows and columns. Here is a slightly edited version of your data:

totals                      <- c(195, 134, 38)
stems                       <- c(22,16,9)
group_stem_counts           <- matrix(c(stems, totals-stems),ncol=3,byrow=TRUE)
rownames(group_stem_counts) <- c("stem", "non-stem")
colnames(group_stem_counts) <- c("Group One","Group Two","Group Three")
group_stem_counts
#          Group One Group Two Group Three
# stem            22        16           9
# non-stem       173       118          29

Now you can run your test:

chisq.test(group_stem_counts)
# 
#         Pearson's Chi-squared test
# 
# data:  group_stem_counts
# X-squared = 4.5225, df = 2, p-value = 0.1042
# 
# Warning message:
# In chisq.test(group_stem_counts) :
#   Chi-squared approximation may be incorrect

This yields the warning that you saw. As a rule of thumb, it is generally recommended that the expected count for each cell under the null hypothesis to be at least 5. However, it has been shown that this is overly conservative, and the chi-squared test is robust even if that isn't exactly the case. We can examine your expected counts like so:

chisq.test(group_stem_counts)$expected
#          Group One Group Two Group Three
# stem      24.97275  17.16076    4.866485
# non-stem 170.02725 116.83924   33.133515
# Warning message:
# In chisq.test(group_stem_counts) :
#   Chi-squared approximation may be incorrect

Your minimum expected count is 4.866485, and all the others are >5. Realistically, this is nothing to bother over. However, if you are concerned about it, you can just simulate the p-value instead of using the chi-squared approximation. Here is the chi-squared test using that option:

chisq.test(group_stem_counts, simulate.p.value=TRUE)
# 
#         Pearson's Chi-squared test with simulated p-value (based on 2000
#         replicates)
# 
# data:  group_stem_counts
# X-squared = 4.5225, df = NA, p-value = 0.1184

As you can see, the p-value is essentially the same.