13

For a given set of data, spread is often calculated either as the standard deviation or as the IQR (inter-quartile range).

Whereas a standard deviation is normalised (z-scores, etc.) and so can be used to compare the spread from two different populations, this is not the case with the IQR since the samples from two different populations could have values at two quite different scales,

 e.g. 
 Pop A:  100, 67, 89, 75, 120, ...
 Pop B:  19, 22, 43, 8, 12, ...

What I'm after is a robust (non-parametric) measure that I can use to compare the variation within different populations.

Choice 1: IQR / Median -- this would be by analogy to the coefficient of variation, i.e. to $ \frac{\sigma}{\mu}$.

Choice 2: Range / IQR

Question: Which is the more meaningful measure for comparing variation between populations? And if it is Choice 1, is Choice 2 useful for anything / meaningful, or is it a fundamentally flawed measure?

Assad Ebrahim
  • 1,357
  • 2
  • 11
  • 22
  • Thanks for the very helpful discussion. Some useful follow-ups -- different definitions of quartiles and hence IQR (John), standard deviation not in fact standardising (Harvey), and QQ plots as a tool to compare two distributions (Peter). (+1 to all three answers!) – Assad Ebrahim Oct 04 '12 at 21:07

6 Answers6

17

It's important to realize the minimum and maximum are often not very good statistics to use (i.e., they can fluctuate greatly from sample to sample, and don't follow a normal distribution as, say, the mean might due to the Central Limit Theorem). As a result, the range is rarely a good choice for anything other than to state the range of this exact sample. For a simple, nonparametric statistic to represent variability, the Inter-Quartile Range is much better. However, while I see the analogy between IQR/median and the coefficient of variation, I don't think this is likely to be the best option.

You may want to look into the median absolute deviation from the median (MADM). That is: $$ MADM = \text{median}(|x_i-\text{median}(\bf x)|) $$ I suspect a better nonparametric analogy to the coefficient of variation would be MADM/median, rather than IQR/median.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • 1
    Interesting choice of `MADM/median`, essentially the middle difference from the middle value. Let's call this Choice 3. Agree with your assessment of Choice 1, so it's out, thanks. When you suggest 'better', what attributes might one use to compare Choice 2 against Choice 3 to see which is better? – Assad Ebrahim Oct 06 '12 at 04:08
  • 1
    The attributes you would use would depend on what your goals for the metric are. However, I only meant that it is a better *analogy* for the CoV. NB that the 3rd quartile is the median of your data that are above the median, & the 1st q is the median of those below, so IQR/2 in the long run will equal MADM (nb, they aren't guaranteed to be equal in a given sample). The IQR will vary further, on ave, from it's true value in the pop, but I'm not sure what, if any, implications that would have, & the stand. err. of IQR/2 should be the same as SE of MADM. – gung - Reinstate Monica Oct 06 '12 at 23:58
  • I see, thanks for the clarification. Good point about the median interpretation of Q3 and Q1. I'll give `MADM/median` a try alongside `IQR/median`. The side-by-side comparison may be interesting. (+1 for the interesting suggestion) – Assad Ebrahim Oct 07 '12 at 00:08
14

The question implies that the standard deviation (SD) is somehow normalized so can be used to compare the variability of two different populations. Not so. As Peter and John said, this normalization is done as when calculating the coefficient of variation (CV), which equals SD/Mean. The SD is in in the same units as the original data. In contrast, the CV is a unitless ratio.

Your choice 1 (IQR/Median) is analogous to the CV. Like the CV, it would only make sense when the data are ratio data. This means that zero is really zero. A weight of zero is no weight. A length of zero is no length. As a counter example, it would not make sense for temperature in C or F, as zero degrees temperature (C or F) does not mean there is no temperature. Simply switching between using C or F scale would give you a different value for the CV or for the ratio of IQR/Median, which makes both those ratios meaningless.

I agree with Peter and John that your second idea (Range/IQR) would not be very robust to outliers, so probably wouldn't be useful.

Harvey Motulsky
  • 14,903
  • 11
  • 51
  • 98
  • 2
    Harvey - thanks -- you're right, *SD* isn't normalised at all... I was confusing the concept of `z-scores` for standardizing values and normalising their position within a distribution in terms of the mean and standard deviation, with this problem, which is about being able to rank groups of products in order of their variability. Choosing your answer as the correct one because while Peter and John's were both very helpful, yours alerted me to the conceptual mixup. Good point on Choice 1 being of limited use near median 0. Fortunately, in my problem, I don't have to worry about this. – Assad Ebrahim Oct 04 '12 at 21:03
  • I'd like to use this in a paper. Is there a good place it's referenced (book/somewhere peer-reviewed)? – Ben Bolker Apr 01 '19 at 20:04
6

I prefer not to compute measures like CV because I almost always have an arbitrary origin for the random variable. Concerning the choice of a robust dispersion measure it is difficult to beat Gini's mean difference, which is the mean of all possible absolute values of differences between two observations. For efficient computation see for example the R rms package GiniMd function. Under normality, Gini's mean difference is 0.98 as efficient as the SD for estimating dispersion.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
6

"Choice 1" is what you want if you are using non-parametrics for the common purpose of reducing the effect of outliers. Even if you're using it because of skew that also has the side effect of commonly having extreme values in the tail, that might be outliers. Your "Choice 2" could be dramatically affected by outliers or any extreme values while the components of your first equation are relatively robust against them.

[This will be a little dependent upon what kind of IQR you select (see the R help on quantile).]

John
  • 21,167
  • 9
  • 48
  • 84
  • You're right, I should have said "this is *analagous* to the definition of the coefficient of variation... (fixed now in the question)! – Assad Ebrahim Oct 04 '12 at 20:19
  • Thanks for the comment *dependent on what kind of IQR you select...* -- I hadn't realised there were so many possibile definitions for quartiles / quantiles! I'm using Excel's built-in `quartile( )` function, and then taking `IQR := Q3 - Q1`. My numbers come from a time series of weekly measurements over a year. The measurements are industrial performance measures and so are from a continuous distribution. The different populations are different product groups. In this situation, I don't think the different definitions would be much different in practice? – Assad Ebrahim Oct 04 '12 at 20:45
3

Like @John I have never heard of that definition of coefficient of variation. I wouldn't call it that if I used it, it will confuse people.

"Which is most useful?" will depend on what you want to use it for. Certainly choice 1 is more robust to outliers, if you are sure that is what you want. But what is the purpose of comparing the two distributions? What are you trying to do?

One alternative is to standardize both measures and then look at summaries.

Another is a QQ plot.

There are many others as well.

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
  • Good point -- should have said *analogous* to the coefficient of variation (I've made the correction). – Assad Ebrahim Oct 04 '12 at 20:49
  • My numbers come from a time series of weekly measurements over a year. The measurements are industrial performance measures and so are from a continuous distribution. The different populations are different product groups and I've got about 50 product groups. What I'm trying to do is be able to compare the inherent variability between different product groups. In particular, I want to be able to rank the product groups in decreasing order of variability. – Assad Ebrahim Oct 04 '12 at 20:50
  • What do you mean 'standardize both measures and then look at summaries'? I thought Choice 1 was standardizing them...! – Assad Ebrahim Oct 04 '12 at 20:55
3

This paper presents two good robust alternatives for the coefficient of variation. One is the interquartile range divided by the median, that is:

IQR/median = (Q3-Q1)/median

The other is the median absolute deviation divided by the median, that is:

MAD/median

They compare them and conclude generaly speaking the second is a little less variable and probably better for most applications.

Armando
  • 131
  • 2