0

The sample standard deviation and standard error of the mean are biased estimators for their corresponding population population parameters. As explained here for the sample standard deviation this is due to the fact that the expectation of a square root does not equal the square root of an expectation, as can be shown by Jensens equality.

Nevertheless, we are inclined to give an interpretation to standard error estimates, in the sence that they give the mean distance of all sample estimators (means) to the population parameter (mean). However, this seems only to be legitimate if the bias is commonly small or vanishes in large samples under quite general conditions (e.g., distributions). In the proof references above, this seems to be the case for normal data. Is this generally so?

Furthermore: the sample standard error is commonly used as estimator of population standard error in a t-statistic. $$t=\frac{\hat{\theta}-\theta}{s.e.({\hat{\theta}})}$$. Is it true that we would actually need an unbiased estimator of the standard error in this formula or is the use of a biased estimator allowed. What is the role of the sample size again (for normal / non-normal data)?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
tomka
  • 5,874
  • 3
  • 30
  • 71
  • 1
    Many useful estimators are biased. Why would unbiasedness be such a big deal? In any case, while it's easy to unbias variance even without knowing the distribution, unbiasing standard deviation depends on the distribution the data are from, and the formula is complicated... and there's no obvious reason you need it [and if you unbiased the estimate of standard error in your t-statistic, you'd have to work out new tables for your t-statistics because they'd no longer be t-distributed. ]. – Glen_b Nov 02 '14 at 15:10
  • It seems to me that the s.e. is only useful as a means to estimate t-statistics, which have known distributions. If they have bias of unknown size, why should I use the estimated standard error to judge the average distance of sample means from population means? That would only make sense if the bias decreases with n, for example. – tomka Nov 02 '14 at 15:15
  • 2
    Of course the bias decreases with $n$. The usual estimator of $\sigma$ is *[consistent](http://en.wikipedia.org/wiki/Consistent_estimator#Bias_versus_consistency)*. You might find [this article](http://en.wikipedia.org/wiki/Bias_of_an_estimator), and some of the other articles it links to, helpful. – Glen_b Nov 02 '14 at 15:32

0 Answers0