Chi square statistic and contingency coefficients

Question

I need some advice on explaining $\chi^2$ as a contingency measure from a pedagogical perspective.

I usually introduce the individual terms in the $\chi^2$ sum as scaled deviations from independence; my argument claims that $d_{ij} = (n_{ij} - E_{ij})^2$ is an absolute measure but we want relative measures that are independent of the units so we should divide by $E_{ij}$. However, the natural question raised is why we do not divide by $E_{ij}^2$ which would truly normalize the counts in the numerator.

In addition, it is difficult to find good, intuitive explanations for the final form of the contingency coefficient which is $\sqrt{\chi^2/(\chi^2 + N)}$

Answer 1

There are two standard accounts of $\chi^2$ as applied to a multinomial distribution, both of which show why the denominator should not be squared:

1. After expanding the logarithm through second order and doing some algebra, the likelihood ratio statistic for the multinomial distribution, $-2\log(l) = 2\sum_{i=1}^k n_i \log(n_i / e_i)$, equals $\chi^2(1 + O(n^{-1/2}))$, so at least asymptotically, $\chi^2$ gives a likelihood ratio test. (I have replaced the double indexing by a single index, without any loss of generality.)

2. Fisher showed that the multinomial distribution arises as the conditional distribution of $k$ independent Poisson variates of intensities $e_{i}$, conditioned on their sum, $n$. Because the variances are $e_{i}$, the standardized values $(n_i - e_i)/\sqrt{e_i}$ are asymptotically Normal for large $n$, whence $\chi^2$ approaches the sum of squares of $k$ independent Normals, subject to the single condition $n_1 + \cdots + n_k=n$, giving it $k-1$ degrees of freedom.

Both arguments clearly show the sense in which $\chi^2$ is valid only asymptotically.

Accounts of both these points of view are found in chapter 30 of Stuart & Ord, Kendall's Advanced Theory of Statistics, Fifth Edition (1987).

I do not know the historical answer to the second question--the contingency coefficient $C$ goes back to Pearson over 100 years ago--but clearly $C$ is an analog of an absolute correlation coefficient, computed as the root of (residual variance / total variance). Indeed,

$$\chi^2 + n = \sum\frac{(n_i-e_i)^2 + e_i^2}{e_i} = \sum\frac{n_i^2}{e_i},$$

which looks exactly like an inverse variance-weighted total sum of squares. In effect, $n$ is the variance "explained" by the fit and $\chi^2$ is the residual variance. Note that $\chi^2$ small implies the fit is good with $C \approx 0$, corresponding to almost no association in a contingency table, while $\chi^2$ large implies the fit is terrible, corresponding to almost perfect association with $C \approx 1$.

Answer 2

I don't know what is "more true" or "less true" form of normalization. Any normalization that makes a particular sense will do. The quantity $(n_{ij}-E_{ij})^2/E_{ij}$ is called squared standardized residual in a cell, and overall Chi-square of a table is the sum of those across all cells. Chi-square can be viewed therefore as the weighted euclidean distance (weight$=1/E_{ij}$) between observed and expected frequencies [this notion of chi-square as the distance is realized in correspondence analysis].

Also, standardized residual follows Poisson distribution, and with large $E_{ij}$ approaches normal distribution. This opens the possibility to infer whether the residual in a cell is significantly large.

Standardized residual is not the only way to normalize residual in a cell. Adjusted residual is another useful form.

Answer 3

An intuitive explanation for why $$ d_{ij}=(n_{ij}−E_{ij})^2/E_{ij}^2 $$ would not be a good test statistic is to notice that it doesn't scale with the size of the sample: Suppose we increase our sample by 10 fold. Then if the null-hypothesis is false, we should be much more likely to reject -- that is, the test statistic $d_{ij}$ should be much larger. Yet if we multiply $n_{ij}$ and $E_{ij}$ by 10 (to get an idea of the scale of the new $d_{ij}$), we see that the value of $d_{ij}$ doesn't change.

To summarize, even though $d_{ij}$ is a very fine and intuitive measure of the difference between two probability distributions, it doesn't reflect anything about how confident we are that the sample distribution is different from the null hypothesis.

Answer 4

About the contingency coefficient, I want to take the opportunity to leave a remark.

Consider the "theoretical chi-square statistic" $$ v= \sum_{i,j}\frac{(p_{i,j}-p_ip_j)^2}{p_ip_j} $$ and the theoretical contigency coefficient $$ C_0 = \sqrt{\frac{v}{v+n}}. $$ It is called the effect size in power calculations. As far as I know, there is no general and precise definition of the effect size. However, considering $C_0$ as the effect size yields something similar to the power of $F$-tests in linear models, which are exact (non-asymptotic) likelihood ratio tests up to an elementary transformation.

Indeed, denoting by $E$ the effect size for such a test (for example $E=\mu/\sigma$ in the case of a simple Gaussian ${\cal N}(\mu, \sigma^2)$ sample and $H_0\colon\{\mu=0\}$), the test statistic under $H_1$ is a non-central $F$-distribution with non-centrality parameter $\boxed{\lambda=nE^2}$. In this situation we do not perform an asymptotic test because it is possible to get the exact law of the test statistic $F$. Asymptotically, $F \approx df_1\times\chi^2$ where $\chi^2$ is the test statistic of the asymptotic likelihood-ratio test. If we used the asymptotic likelihood ratio instead of the exact $F$-test, we would get a non-central Chi-square under $H_1$ instead of a non-central $F$ distribution : $F_{df_1,df_2}(\lambda) \approx \frac{1}{df_1}\chi^2_{df_1}(\lambda)$ when $df_2$ is large.

Similarly, in the context of contingency tables, when $n$ is large, the $\chi^2$ test statistic under $H_1$ is approximated by a non-central $\chi^2$ distribution with non-centrality parameter $\boxed{\lambda=n C_0^2}$, as we can check by this example of power calculation:

C0 <- 0.3 # theoretical effect size
n <- 100 # total count
alpha <- 0.05 # type I error
#--- Power calculcation with the pwr package ---#
library(pwr); pwr.chisq.test(C0, N=n, df=1, sig.level=alpha)
## 
##      Chi squared power calculation 
## 
##               w = 0.3
##               N = 100
##              df = 1
##       sig.level = 0.05
##           power = 0.8508388

#--- Direct power calculation ---#
threshold <- qchisq(1-alpha, df = 1) # the critical value of the test statistic
lambda <- n*C0^2 # non-centrality parameter
1 - pchisq(threshold, df = 1, ncp = lambda) # we get the same result
## [1] 0.8508388

Thus each of these examples ($F$-tests in Gaussian linear models and $\chi^2$ tests for contingency tables) asymptotically give a likelihood ratio test (see @whuber's answer about the Pearson $\chi^2$), and their test statistic under $H_1$ is approximated, for large $n$, by a non-central $\chi^2$ distribution with non-centrality parameter $\boxed{\lambda=n\times\text{effect size}^2}$.