4

I'm having difficulty figuring out the difference between (typically referred to as S.E):

A. The Standard Error

And

B. The Standard Deviation of the Sample Mean (typically referred to as s)

Are they the same thing?

OR

  1. S.E is the standard deviation of the the mean of the sampling distribution similar formula as the standard deviation except you use n-1 instead of n in the denominator
  2. s is the standard deviation of a single random sample -- same formula as the standard deviation
user1172468
  • 1,505
  • 5
  • 21
  • 36

4 Answers4

9

The standard deviation of the mean is usually unknown. We would write it as $$ \sigma_{\bar x } ={\sigma \over \sqrt n} $$

The standard error of the mean is an estimate of the standard deviation of the mean. $$ \hat \sigma_{\bar x} = {s \over \sqrt n}. $$

soakley
  • 4,341
  • 3
  • 16
  • 27
  • Does the first formula even make sense ? if I have the population standard deviation and it is a constant number , what does exacty does the 1st formulat tell me then ? – Oleg Mar 21 '17 at 10:45
  • 2
    Yes, it does make sense. Remember that the sample mean $\bar x$ is itself a random variable. So the first formula tells you the standard deviation of the random variable $\bar x$ in terms of the standard deviation of the original distribution and the sample size. – soakley Mar 21 '17 at 17:21
  • 1
    1.so it's like saying that by taking the SD of the means of all our samples we get the SEM. 2.This SEM will show us how far away is our "mean of means" (mean of all our sample means) from the TRUE population mean. – Oleg Mar 21 '17 at 20:55
  • 2
    You only need one sample to estimate the SEM. That is done using the second formula. Typically there is no "mean of all our sample means" because, again, you are using only one sample. That's the beauty of the second formula - you are able to estimate the uncertainty in your sample mean even though you only have data from a single sample. – soakley Mar 22 '17 at 18:12
  • 1
    but if I choose 2 different samples I from the same random var , I could also get 2 different standard errors of the mean , wouldn't that be a problem ? – Oleg Mar 22 '17 at 18:26
  • 2
    Not a problem. If I take a sample of size 10 and you take an independent sample of size 10, it is not unusual if we get different values of our sample mean. It's no different for the SEM. The SEM is a random variable (because it is a function of $s,$ which is a random variable) so we can expect it to take on different values for different samples. – soakley Mar 22 '17 at 22:11
  • @soakley Since SEM depends on s and it can have different values for different samples, why do we use it to calculate confidence interval. Why does it work? Where did it come from? I have seen proof for the first formula,and it makes sense. The second formula doesnt make any sense. – bluejimmy Feb 08 '20 at 21:49
  • Why do we use $\bar x$ since it also has different values for different samples? We use it because in real-world applications we cannot typically know the true value of the mean. So we use an estimate. It's the same for standard deviation (whether it's the population standard deviation or the standard deviation of the mean.) We don't know it, so we need formulas like the second one to be able to estimate it. Get a good textbook to answer your other questions - they are good ones, but a comment here doesn't give them justice. There are other questions/answers on this site than can help as well. – soakley Feb 08 '20 at 22:00
5

A standard error can be computed for almost any parameter you compute from data, not just the mean. The phrase "the standard error" is therefore ambiguous. I assume you are asking about the standard error of the mean.

Here are the key differences between the standard deviation (SD) and the standard error of the mean (SEM)

  • The SD quantifies scatter — how much the values vary from one another.

  • The SEM quantifies how precisely you have determined the true mean of the population. It takes into account both the value of the SD and the sample size.

  • Both SD and SEM are in the same units -- the units of the data.

  • The SEM, by definition, is always smaller than the SD.

  • The SEM gets smaller as your samples get larger. This makes sense, because the mean of a large sample is likely to be closer to the true population mean than is the mean of a small sample. With a huge sample, you'll know the value of the mean with a lot of precision even if the data are very scattered.

  • The SD does not change predictably as you acquire more data. The SD you compute from a sample is the best possible estimate of the SD of the overall population. As you collect more data, you'll assess the SD of the population with more precision. But you can't predict whether the SD from a larger sample will be bigger or smaller than the SD from a small sample. (This is not strictly true. It is the variance -- the SD squared -- that doesn't change predictably, but the change in SD is trivial and much much smaller than the change in the SEM.)

  • The SEM is hard to define conceptually. The only real "purpose" of an SEM is as an "ingredient" to compute the confidence interval of the mean.
  • The SEM is computed from the SD and sample size (n) as $$SEM ={SD \over \sqrt n}. $$

(From the GraphPad statistics guide that I wrote.)

Harvey Motulsky
  • 14,903
  • 11
  • 51
  • 98
1

The official term for the dispersion measure (of a distribution, of a sample etc) is "standard deviation" - the square root of the variance.

The tern "standard error" is more often used in the context of a regression model, and you can find it as "the standard error of regression". It is the square root of the sum of squared residuals from the regression - divided sometimes by sample size $n$ (and then it is the maximum likelihood estimator of the standard deviation of the error term), or by $n-k$ ($k$ being the number of regressors), and then it is the ordinary least squares (OLS) estimator of the standard deviation of the error term.

So you see that they are closely related, but not the same thing.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
1

For normally distributed data, the SE = s, as the mean is an explicit parameter of the normal distribution. In general, standard error arises in Likelihood theory, where you are forming inferences from a likelihood function as opposed to the true sampling distribution. For example, if you are modeling some data as iid Exponential then you would form the likelihood function for your data $L(X|\lambda)= \prod L_{exp}(x_i|\lambda)$, with unknown $\lambda$ and then optimize L(X|$\lambda$) for $\lambda^*$ (i.e, maximum likelihood estimator). The standard error is defined as the curvature of the quadratic approximation to log(L(X|$\lambda^*$))at the MLE, which will equal the standard deviation for normal data. the only difficulty is that for non-normal data, you will need to do a second step to transform the actual parameters of your distribution (e.g., $\lambda$) into an estimate of the sample mean. Here, you would map $\frac{1}{\lambda} \rightarrow\mu$, so the likelihood of the latter equals that of the former, then take the log of that likelihood and get a standard error of that transformed likelihood function. Sorry for the long answer, but its not super clear cut in all cases. Sometime, its even used loosely, so yo need to read the documentation to really know.