Is a $\chi^2$ test appropriate in this case?

Question

I have a sample $X_1, X_2,\cdots,X_n$ (where $n$ is small), where we believe theoretically that each $X_i$ follows a binomial distribution $B(m,p_i)$, I want to test if all the $p_i$s have the same value $p_0$ (which is known à priori) or not. Is the $\chi^2$ test appropriate in this case?

EDIT: Here's what I want to test:

          square   round      triangle
red      10          10           20
blue     20          60           30
p_∙,j    10/30=0.3 10/70=0.14     20/50=0.4

I want to test whether $p_{\bullet,1}=p_{\bullet,2}=p_{\bullet,3}=0.5$ or not.

Answer 1

Short answer is yes, it is. However, it will only test the null hypothesis that all parameters are equal. If you are interested which of the $p_i$ are different, you will need to run a post hoc test.

EDIT: OK, I see now where the confusion is. $\chi^2$ tests the goodness of fit, but one of the most common application is testing for dependence. In a way this is exactly the same thing, but commonly one shows it as a contingency table.

We have two variables, $X$ (e.g. "round vs square" or "0 vs 1") and $Y$ (e.g. red vs blue). This gives us $X \times Y$ bins (e.g. four bins: round red, round blue, square red and square blue), which can be neatly represented as a contingency table. Each cell contains the number of events in that cell.

       square      round
red     10          40
blue    20          40

How does this relate to your question? Under independence, we assume that the probabilities of getting a "red" outcome is the same for "squares" and "rounds". We assume that "squares" and "rounds" follow a binomial distribution with the same parameter $p$. This is the null ($H_0$); rejecting it means that there is a dependence between $X$ and $Y$ -- that is, $p$ is different for different $X_i$.

Once again -- "square" and "round" is $X_1$ and $X_2$ from your question, it each follows binomial distribution (with results "red" or "blue") and we want to test, using $\chi^2$, whether the probability of getting "blue" is the same for "squares" as for "rounds".

To do that, we are first proposing a theoretical distribution of the values based on $p_{i,\bullet}$ and $p_{\bullet,j}$ (row-wise and column-wise proportions). In our example,

       square      round   p_i,∙
red     10          40     50/110 = 0.45
blue    20          40     60/110 = 0.55
p_∙,j   30/110=0.27 80/10=0.73

O.45 is the experimental probability of getting a "red" under independence; this is the $p$ from your question, but same for "squares" and "rounds" -- same for all $X_i$ (remember, this is our null!).

Note that the four types of events (four bins) are disjoint and their probabilities obviously add up to 1, just as you have described it for the goodness of fit. Each cell in the contingency table is your $A_{i,j}$, and $\bigcup\limits_{i,j} A_{i,j} = \Theta$, and $A_{i,j} \cap A_{m,n} = \emptyset$ if $i \neq m$ or $j \neq n$.

Assuming independence, the expected numbers of observations in each cell will be $N\cdot p_{\bullet,j} \cdot p_{i,\bullet}$.

This is gives the distribution, and we test how good the data fits it. For this we use $\chi^2$. If we reject the null, we infer that the independence assumption is not fulfilled, which means that some of the $p_i$ corresponding to $X_i$ are different. Precisely what you wanted to test.

Answer 2

OK, so here is one way to answer your question. Suppose that you observe: $$ X_1,...,X_i,...,X_k\sim Bin(n_i,\pi_i) $$ and you would like to test for whether: $$ H_0:\pi_1 = \dots = \pi_k = \pi_0. $$ There are several ways that this could be done, but I think that the following might work well for testing whether at least one is zero. For each variable compute a test statistics: $$ T_i = 2\log\frac{\bar{X}_i^{X_i}(1 - \bar{X}_i)^{n_i - X_i}} {\pi_0^{X_i}(1 - \pi_0)^{n_i - X_i}} \sim \chi^{2}_1. $$ For each test statistic compute a p-value according to the $\chi^{2}_1$ distribution to obtain $k$ p-values: $$ p_1,\dots,p_k. $$ Now, apply Benjamini-Hochberg False discovery rate adjustment to the p-values to obtain q-values: $$ q_1,...,q_k. $$ A nice thing about the BH adjustment is that it can serve as a global test for whether at least one proportion is different from $\pi_0$. So if: $$ \min_i q_i < \alpha, $$ you can reject the null hypothesis that all propensities are equal to $\pi_0$.

A second option for a global test is to sum the test statistics: $$ \sum_{i=1}^{k} T_i \sim \chi^{2}_k. $$

Here is a toy example for implementation:

manyBinomTest <- function(x, n, p0) {
  props <- x / n
  altlik <- dbinom(x, n, x / n, log = TRUE)
  nulllik <- dbinom(x, n, p0, log = TRUE)
  testStats <- 2 * (altlik - nulllik)
  pvals <- pchisq(testStats, 1, lower.tail = FALSE)
  qvals <- p.adjust(pvals, method = "BH")
  globalTest <- sum(testStats)
  globalPval <- pchisq(globalTest, df = length(x), lower.tail = FALSE)
  return(list(qvals = qvals, global = globalPval))
}

p0 <- 0.3
n <- 100
k <- 200

reps <- 10^3
qglobal <- numeric(reps)
pglobal <- numeric(reps)
for(i in 1:reps) {
  x <- rbinom(k, n, p0)
  result <- manyBinomTest(x, n, p0)
  qglobal[i] <- min(result$qvals)
  pglobal[i] <- result$global
}

par(mfrow = c(1, 2))
qqplot(qunif(ppoints(reps)), qglobal,
       main = "test via BH screening")
abline(a = 0, b = 1, col = "red")

qqplot(qunif(ppoints(reps)), pglobal,
       main = "test via likelihood ratio")
abline(a = 0, b = 1, col = "red")

Answer 3

By definition the $\chi^2$ random variable with $n$ degrees of freedom is a sum of $n$ squared independent standard normal variables, i.e. if $X_i \sim N(0,1)$ then $X^2=\sum_{i=1}^n X_i^2$ is $\chi^2(n)$.

It is also known that a binomial variable of size $N$ and success probability $p$ can be approximated by a normal random variable with mean $Np$ and standard deviation $\sqrt{Np(1-p)}$. The approximation works well if p is close to 50% for reasonable sizes $N$.

So if the approximation works well and $B_i \sim Bin(N,p)$ and independent then $X_i=\frac{B_i-Np}{\sqrt{Np(1-p)}}$ is standard normal.

Therefore $\sum_{i=1}^n \left( \frac{B_i-Np}{\sqrt{Np(1-p)}}\right)^2$ is approximately $\chi^2(n)$ and can be used as test statistic in a chi-square test.