30

I read from there that the standard error of the sample variance is

$$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$

What is the standard error of the sample standard deviation?

I'd be tempted to guess and say that $SE_{s} = \sqrt{SE_{s^2}}$ but I am not sure.

Remi.b
  • 4,572
  • 12
  • 34
  • 64
  • 1
    You mean the standard error of the _sample_ variance/standard deviation I guess? If yes, any particular distribution in mind? – Alecos Papadopoulos Jun 11 '15 at 15:46
  • Yes, this is what I meant. I edited my post in reaction to your comment thanks. I am surprised that you're asking what distribution I have in mind. I would have not expect it matters. No I don't have any particular distribution in mind. The population form which my sample is taken is likely not normal. It is probably slightly skewed and has very long tails. – Remi.b Jun 11 '15 at 15:48
  • 2
    Asymptotically it "does not matter". In finite samples it certainly does. For the asymptotic answer see http://stats.stackexchange.com/a/105338/28746 – Alecos Papadopoulos Jun 11 '15 at 15:52
  • 1
    And next you ask for the standard error of the standard error of the standard error ... – kjetil b halvorsen Jun 11 '15 at 15:52
  • @kjetilbhalvorsen Is this a decreasing or an increasing sequence? – Alecos Papadopoulos Jun 11 '15 at 15:53
  • 7
    @Kjetil Your thought is an amusing one. Please note, though, that the SE as defined here is not a random variable; it has no standard error. One often *estimates* the SE by using an estimate of $\sigma^4$ and frequently--by a conventional abuse of language--still calls that *estimated* SE a "standard error." As such it is indeed a random variable and will have a standard error. I'm sure you're aware of the distinction (and had it in mind when you wrote your comment), but I want to emphasize it so that people don't misunderstand the original question as a result of pondering your comment. – whuber Jun 11 '15 at 16:04
  • The following paper gives the standard error for different statistics. https://www.researchgate.net/publication/285590690_Standard_errors_A_review_and_evaluation_of_standard_error_estimators_using_Monte_Carlo_simulations – Satheesh Murugesan Jan 13 '21 at 13:58

1 Answers1

35

Let $\mu_4 = E(X-\mu)^4$. Then, the formula for the SE of $s^2$ is:

$$ se(s^2) = \sqrt{ \frac{1}{n}\left(\mu_4 -\frac{n-3}{n-1} \sigma^4\right)} $$ This is an exact formula, valid for any sample size and distribution, and is proved on page 438, of Rao, 1973, assuming that the $\mu_4$ is finite. The formula you gave in your question applies only to Normally distributed data.

Let $\hat{\theta} = s^2$. You want to find the SE of $ g(\hat{\theta})$, where $g(u) = \sqrt{u}$.

There is no general exact formula for this standard error, as @Alecos Papadopoulos pointed out. However, one can drive an approximate (large sample) standard error by means of the delta method. (See Wikipedia entry for "delta method").

Here's how Rao, 1973, 6.a.2.4 put it. I include the absolute value indicators, which he incorrectly omitted.

$$ se(g(\hat{\theta})) \approx |g'(\hat\theta)|\times se(\hat{\theta}) $$ where $g'$ is the first derivative.

Now for the square root function $g$

$$ g'(u) = \frac{1}{2\thinspace u^{1/2}} $$

So:

$$ se(s)\approx \frac{1}{2 \sigma} se(s^2) $$

In practice I would estimate the standard error by the bootstrap or jackknife.

Reference:

CR Rao (1973) Linear Statistical Inference and its Applications 2nd Ed, John Wiley & Sons, NY

Steve Samuels
  • 2,497
  • 11
  • 10
  • 1
    +1 It's nice to see these results so clearly laid out and explained. Although I don't have Rao 1973 in front of me, I would expect the multiplicative factor in his formula ought to be $|g^\prime(\hat\theta)|$, for otherwise you would conclude that any order-reversing transformation would have a negative standard error. – whuber Jun 17 '15 at 14:55
  • 1
    Thanks. You are right about the absolute value. Rao had omitted it (equation 6.a.2.4 in both the 1968 and 1973 editions.) .The proof of the delta method is really for the variance, where the multiplier is [g']^2. – Steve Samuels Jun 17 '15 at 15:37
  • whats the bootstrap and jackknife? – alpha_989 Sep 16 '17 at 22:27
  • 1
    @alpha_989 The [bootstrap](https://en.wikipedia.org/wiki/Resampling_(statistics)#Bootstrap) and [jackknife](https://en.wikipedia.org/wiki/Resampling_(statistics)#Jackknife) methods use resampling in order to estimate precision. They are useful because you don't need to do the error propagation by hand. – Ben Jones Jan 15 '19 at 17:03