2

So, if there are $N$ data points in my sample space of any randomly distributed variable $X$. The standard deviation, $\sigma$, (from my understanding) is the root mean squared of the error (from the mean, $\bar{X}$,) which leads to the formula shown below $$\sigma = \sqrt{\frac{\sum_{i=1}^N\left(X_i-\bar{X}\right)^2}{N}}$$

But, in certain cases, the deviation is shown using the formula

$$\sigma = \sqrt{\frac{\sum_{i=1}^N\left(X_i-\bar{X}\right)^2}{N-1}}$$

For example, in this link (wikipedia). I do not really understand how this helps to reduce bias, as said in the above link.

Steffen Moritz
  • 1,564
  • 2
  • 15
  • 22
ubuntu_noob
  • 327
  • 3
  • 8
  • 3
    Long story short: the first formula is used to calculate the population variance, the second formula is used to estimate the population variance from a sample. Have a look at https://stats.stackexchange.com/questions/31036/what-is-the-difference-between-a-consistent-estimator-and-an-unbiased-estimator/ – user2974951 Sep 11 '19 at 14:05
  • There is little to nothing to add to the comment of @user2974951 . For large N the difference between both becomes negligible. But in real world use of statistics (e.g. in physics) there is a sometimes significant difference. Most introductory books on statistics include some discussion and derivation. Eg. Statistics by R. Barlow (1989) – cherub Sep 11 '19 at 14:22

1 Answers1

2

It depends on how you want to calculate variance. Remember that you're doing inferential statistics, so there is some unknown true variance of the population distribution, and we perform calculations to try to estimate that true value.

There are many ways to calculate estimators. A common method is called maximum likelihood estimation (MLE). Using that approach, we get that $\hat{\sigma}_{MLE}^2 = \dfrac{\sum_{i=1}^n(x_i - \bar{x})^2}{n}$. Since this so closely resembles the variance calculation for a population (the average of the squared deviations from the mean), this is sometimes called the population variance formula.

However, we get this annoying feature of $\hat{\sigma}_{MLE}^2$ that its expected value (average) is not the population parameter $\sigma^2$. To correct for that, we divide by $n-1$ instead of just $n$, and then the expected value of $\hat{\sigma}^2 = \dfrac{\sum_{i=1}^n(x_i - \bar{x})^2}{n-1}$ is the population parameter $\sigma^2$.

In statistical terminology, the $\hat{\sigma}_{MLE}^2$ estimator is biased while $\hat{\sigma}^2$ is unbiased. All else equal, we would prefer an unbiased estimator.

Amazingly, when you take the square root of either estimator, you get a biased estimator of the standard deviation. There is not a formula for an unbiased estimator of standard deviation that works in general, though it has been worked out for a normal distribution and probably some other common distributions, but it would be a different formula for a normal population than an exponential population.

As a heads up, when people don't specify which estimators they are using, they are assumed to be using the unbiased estimator for variance and its square root for standard deviation (unless you know your field to deviate from this).

Dave
  • 28,473
  • 4
  • 52
  • 104
  • The consideration in the last paragraph has always amazed me... I can understand how applying a non-linear function alters bias mathematically, but it always sounded weird that estimators for variance and standard error can't both be unbiased and agree with each other at the same time! – polettix Sep 11 '19 at 15:21