3

Given a set of data, it seems the intuitive way to measure the deviation of the data is the following. $$ \frac{1}{n}\sum_x |x_i-\bar{x}| $$ But I understand that for theoretical reasons it is easier to work with the standard deviation formula. $$ \sqrt{ \frac{1}{n -1} \sum_x (x_i-\bar{x})^2 } $$ It can be said that $$ \sum_x |x_i-\bar{x}| \geq \sqrt{\sum_x (x_i-\bar{x})^2} $$ just as the sum of the two smaller sides of a triangle is larger than the hypotenuse.

The following can also be said. $$ \frac{1}{n} \lt \sqrt{\frac{1}{n-1}} $$ Do these two inequalities compensate for each other such that the standard deviation definition of deviation is equivalent to the intuitive definition of deviation? Or are the two definitions fundamentally different, in which case standard deviation does not correctly reflect our intuitive sense of deviation?

Joseph Johnston
  • 133
  • 2
  • 4
    You may be interested in: [Why square the difference instead of taking the absolute value in standard deviation?](http://stats.stackexchange.com/q/118/7290) – gung - Reinstate Monica Mar 04 '16 at 20:08
  • It may be interesting to note that in the early days of probability, people used the same "obvious" loss function as you: the sum of absolute deviations. Unfortunately, they were not very successful in obtaining nice mathematical formulas as far as parameter estimation is concerned. That is until Gauss found that things would get much easier if squared deviations were used as loss criterium. – StijnDeVuyst Mar 05 '16 at 09:05

1 Answers1

2

[It's not just that the root-mean-square form is easier to work with; it has properties that make it attractive, such as the fact that the variances of sums of random variables have simple form. Though perhaps that's what you were getting at.]

The second inequality doesn't compensate for the first; not even on average.

If we define $s_n=\sqrt{ \frac{1}{n} \sum_i (x_i-\bar{x})^2}$, we can apply Jensen's inequality to see that the mean deviation will never be larger than $s_n$.

That is, if $m_n$ is the mean deviation, $s_n\geq m_n$ (with equality only when they're both 0).

Then we can see that the usual Bessel-corrected standard deviation $s_{n-1}=\sqrt{\frac{n}{n-1}}\,s_n\geq s_n$ (with equality only when they're both 0).

Hence $s_{n-1}\geq m_n$, (again with equality only when they're both 0).

The expected value of the ratio of the mean deviation to standard deviation - (while necessarily between 0 and 1) depends on the distribution you draw from. Indeed the ratio was proposed by Geary as a test of normality in 1935; it works well against symmetric heavy-tailed alternatives.

[For normally distributed data, the expected value of the ratio in large samples is about 0.8]

Glen_b
  • 257,508
  • 32
  • 553
  • 939