5

I am looking for a test which works exactly like the chi-squared test but instead of calculating the test statistic as

  • the sum of squared differences of observed and expected values divided by expected values

it calculates

  • the sum of absolute differences of observed and expected values divided by expected values.

Is there any such test and which distribution would the test statistic follow?

vonjd
  • 5,886
  • 4
  • 47
  • 59
  • 4
    Why are you looking for such a test? What properties do you need it to have? In what sense do you expect your test statistic to improve on the $\chi^2$ statistic? What assumptions will you make about the underlying data distribution? – whuber Mar 29 '15 at 19:21
  • @whuber: The square operator distorts the differences non-linearly and I think the absolute differences would have a more intuitive interpretation when calculating the test statistic. – vonjd Mar 29 '15 at 19:24
  • 4
    They might have a more intuitive interpretation--but they typically have awful characteristics in terms of test size and power. Choosing a test based on one's intuition typically is a poor approach. Instead people begin with a probability model of the data and criteria to help them balance the expected costs of decision errors: these things then *determine* the proper test statistic to use. – whuber Mar 29 '15 at 19:28
  • 1
    @whuber: That is interesting, could you elaborate on "they typically have awful characteristics in terms of test size and power." Perhaps giving some examples or references where I can find more? Thank you. – vonjd Mar 29 '15 at 19:30
  • 3
    Any good book covering hypothesis testing from a sufficiently advanced viewpoint ought to serve as a good reference and source of examples. I often refer to Kiefer's *Introduction to Statistical Inference* for clear explanations (at the undergraduate level of mathematics) or Stuart & Ord's *Kendall's Advanced Theory of Statistics* for more comprehensive analysis (at a beginning graduate level of mathematics). Stuart & Ord have good coverage of the $\chi^2$ test; I used them as a primary resource for my discussion of it at http://stats.stackexchange.com/a/17148, for instance. – whuber Mar 29 '15 at 19:35
  • The division by expected in the chi-square statistic is related to the variance of the numerator. It is used to standardize the square of $(O_i-E_i)$. (The full explanation is slightly more complicated and relates also to the covariances of such terms.) But why are you dividing the $|O_i-E_i|$ by expected? That wouldn't scale the absolute differences. – Glen_b Mar 29 '15 at 21:55
  • Can you say more about what you're trying to achieve? What's the underlying problem you're trying to arrive at a solution for? – Glen_b Mar 30 '15 at 00:17

2 Answers2

7

You don't very clearly explain your concern, but I suppose that you're probably worried about the relative weight the chi-square puts on the cases where $(O_i-E_i)^2$ is large relative to the $E_i$ on the denominator. A single such term can dominate the statistic.

I also assume (at least to start with) that you're asking about the multinomial goodness of fit chi-square.

Note that your statistic is $\sum_{i=1}^k|\frac{O_i-E_i}{E_i}|= \sum_{i=1}^k|\frac{O_i}{E_i}-1|$.

If you want to reduce the effect of the larger differences between observed and expected values for multinomial goodness of fit tests, there's the power-divergence family[1]:

$$2nI^\lambda=\frac{2}{\lambda(\lambda+1)}\sum_{i=1}^k O_i\left\{\left(\frac{O_i}{E_i}\right)^\lambda-1\right\}\,;\,\lambda\in\mathbb{R}$$

Some authors refer to $2nI^\lambda$ as $\text{CR}(\lambda)$.

The choice $\lambda=1$ gives the ordinary chi-square, $\lambda=0$ gives the G test, $\lambda=-\frac{_1}{^2}$ corresponds to the Freeman-Tukey statistic[2][3], and so on.

These all have asymptotic chi-square distributions.

Of those, two that would seem to come more-or-less near to what you were seeking in the statistic (at least in the sense of having a power of $O_i$ near 1) would be $\lambda=0$, the G-test (likelihood ratio test):

$$G = 2\sum_{i=1}^k O_i\cdot\ln\left(\frac{O_i}{E_i}\right)$$

and the (usual form of the) Freeman-Tukey:

$$T^2 = 4\sum_{i=1}^k \left(\sqrt{O_i}-\sqrt{E_i}\right)^2$$

If you're looking for a test for a contingency table, the likelihood ratio test is widely accepted and it has good properties; the distribution of the test statistic tends to work a little better at small sample sizes as well. I've seen at least one paper where power divergence statistics (aside from the usual chi-square and likelihood ratio test) were adapted to the contingency table case, but haven't pursued them.

--

More generally, you can use pretty much whatever statistic you choose if you can sample from the null distribution of your test statistic, but (as whuber points out) you should consider the properties of your choice of statistic. Just choosing statistics at whim may produce poor power characteristics (power may be investigated for specific alternatives of interest).

You should justify your choice of test statistic carefully - why that statistic, rather than some other, similar statistic? e.g. why $\sum_i|\frac{O_i-E_i}{E_i}|= \sum_i|O_i/E_i-1|$ rather than something that would seem to be more natural, perhaps ones such as $\sum_i |\frac{O_i-E_i}{\sqrt{E_i}}|$ or $\sum_i |\frac{O_i-E_i}{\sqrt{E_i(1-\pi_i)}}|$.

Under multinomial sampling under $H_0$ its easy enough to produce random tables of counts and so investigate the distribution of some test statistic under the null (and so produce a test). If you condition on the margins, it's also possible to sample contingency tables of counts under the null of independence (e.g. R has a function to do so).

It's generally better to start with something whose good characteristics are established.

[1] Cressie, N. and Read, T.R.C (1984),
"Multinomial Goodness-of-fit Tests"
JRSSB, 46(3), 440-464

[2] Read, C. B. (1993),
"Freeman—Tukey chi-squared goodness-of-fit statistics"
Statistics & Probability Letters, 18(4), November: 271–278

[3] Freeman, M. F. and Tukey, J. W. (1950),
"Transformations related to the angular and the square root",
The Annals of Mathematical Statistics 21(4): 607–611,
doi:10.1214/aoms/1177729756

Glen_b
  • 257,508
  • 32
  • 553
  • 939
1

Just to add to this, the concern about weighting for chi-square and related tests is well-founded in certain cases. One solution is to use the (unweighted) Euclidean distance. This isn't the absolute distance, but I think it's a little more intuitive than the weighted Euclidean distance involved in the chi-square test.

There's been several recent papers on this, but for a relatively detailed explication, see William Perkins, Mark Tygert, Rachel Ward's paper. The free version is here: https://arxiv.org/abs/1108.4126

num_39
  • 380
  • 1
  • 7