How to quantify variation within one experiment?

Question

Given a couple of measurements I get by running an experiment, I need to express the fact that they are similar to some extent. So, I computed their coefficient of variation, i.e., stdev/mean.

Here is a sample:

3.76025 2.98375 3.78946 3.72195 3.11426 3.60366

and CV = 10.12%

Now I am confused, I don't know how to interpret that number, obviously the lower the better, but what is low? E.g., is 20% and below considered low?

Here is some more of my results: http://pastebin.com/raw.php?i=sg5S05rM
I have thousands of such lines, and all I want to say is that numbers within each line are always more or less similar.

Should I use another measure?

Answer 1

In my experience coefficient of variation is particularly used for variables with positive means. I said "intended" not defined. A CV makes no sense for zero mean random variables. I see it having value especially for random variable that are strictly positive. I could have a similar value for strictly negative variables but I would want it to be the standard deviation divided by the absolute value of the mean. For variable with very small means a very high coefficient of variation does not seem to me to tell you anything. But a high coefficient of variation relative to a large mean does in my view.

So I think determining what is a good low CV really depends on the actual mean. Two variables with 0 mean, one with a small variance and the other with a large one will both have an infinite coefficient of variation and the CV cannot be used to determine similarity.

Also what to call small is inherently subjective. It is not a statistical issue.

Based on the rows of data you provided I have two comments. (1) Although there is not a great deal of data there certainly appears to be similarity across each row. The particular row that had the 10.12% CV seemed to my eye to have the greatest variation of any of the rows. Now the within row variability seems small compared to the variability between rows which is large. Are these differences due to selection of unrelated data or is there some commanality between rows? (2) In the later case a two-way analysis of variance might show a large and statistically significant effect between rows but an insignificant effect across rows. If there is no relationship then maybe each row can be tested for similarity using one-way ANOVAs. Numbers are just number. To give good statistical advice we need to know the meanings of the numbers.

Answer 2

This depends on what you think the scale variance of your number are. If you think that the absolute variation about the mean should depend on the mean (e.g. $10 \pm 1$ is about the same as $100 \pm 10$), then CV is a fine choice because it will control for this. If, however, you think that the amount of variation shouldn't depend on the mean ($10 \pm 1$ is about the same as 100 $\pm$ 1), then dividing by the mean is not desirable, and you should use something like variance or standard deviation ($\sigma$). A 95% confidence interval is given by $\mu \pm 1.96 * \sigma$. You could even use the range (max - min), though that will be more susceptible to outliers.

As far as whether a particular statistic is "big" or "small", that depends completely on what you're measuring and how you feel about it. Standard deviation will be in the same units as your measurements, CV will be a proportion (percent if you divide by 100). You say your data is for time elapse. A 20% CV could be fine for something where you expect wide variation, maybe how long a whale stays underwater, but it sounds very high for something that should be regular. Similarly, a standard deviation of 1 minute for how long it takes me to walk from my office to another building seems reasonable, about the same. If it is 1 hour, I'm less happy about the situation.