1

I have this code here (it's out of a book called "An Introduction to Bootstrap Methods with Applications to R")

We are working with the estimator: $S_n^2=\frac{\sum_{i=1}^{n}(X_i-X_b)^2}{n}$ where $X_b = \frac{\sum_{i=1}^{n}X_i}{n}$, $S_n^2$ is the maximum likelihood estimate of $\sigma^2$. I'm relatively new to this statistics regarding likelihoods and estimators.

set.seed(5^13)
n<-25
x<-rnorm(n) # random sample of unit normal variate
varx<-var(x)*(n-1)/n  # sample variance, uncorrected
c(varx, varx-1.0, -1/sqrt(n)) # sample variance and bias relative to true 
                      # value 1.0 and expected value of bias

and I'm having trouble comprehending the last argument of the last line.

-1/sqrt(n)

Can someone tell me how he got this term that he calls "expected value of bias"?

New edited below

In the book he says that the estimator $S_n^2$ has a bias of $b=-\sigma^2/n$, which I think the -1/sqrt(n) is the expected value of this value $b$. I will have to go and research how the bias of that estimator comes about, however if you have anything to enlighten me on this, it would be greatly appreciated.

I'm familiar with with we divide by (n-1) when estimating population variance from an intuitive and mathematical perspective. He is demonstrating the importance of this by biasing the output of var().

I believe my question is even more simplistic than this, in that I cannot see the connection of -1/sqrt(n) to this. I think it is a basic math thing that I'm not seeing.

enter image description here

Bucephalus
  • 211
  • 1
  • 10
  • 1
    Possible duplicate of [Intuitive explanation for dividing by $n-1$ when calculating standard deviation?](https://stats.stackexchange.com/questions/3931/intuitive-explanation-for-dividing-by-n-1-when-calculating-standard-deviation) – jbowman Oct 03 '18 at 00:37
  • 1
    The $-1/\sqrt{n}$ isn't the expected value of the bias when $\sigma = 1$, it *is* the bias when $\sigma = 1$ (as it does in this case.) – jbowman Oct 03 '18 at 00:38
  • I see, let me investigate this for a moment. I'm sure this is something obvious that i'm not seeing. Thankyou @jbowman – Bucephalus Oct 03 '18 at 00:51
  • Those comments in the code are the authors comments. – Bucephalus Oct 03 '18 at 00:53
  • Oh yeah, i see now, $b=-\sigma^2/n=-\sigma/\sqrt{n}=-1/\sqrt{n}$. I will have to go and do some more reading on properties of estimators I believe. Thanks @jbowman. – Bucephalus Oct 03 '18 at 00:57
  • I can see how it is the bias, not the expected value of the bias. Those comments through me off. Thanks @jbowman – Bucephalus Oct 03 '18 at 00:58
  • Probably seems trivial to you, but can you make it an answer so I can close the question please. @jbowman – Bucephalus Oct 03 '18 at 01:00
  • 1
    @Bucephalus In your comment you say "$b=-\sigma^2/n=-\sigma/\sqrt{n} $". How does that second equality work? – Glen_b Oct 03 '18 at 01:38
  • @Glen_b hhhmmmm, maybe I'm wrong there. I cannot take the square root of a negative number. Once again, I will have to go and read about estimation and bias, as I don't even know where $b=-\sigma^2/n$ comes from yet. – Bucephalus Oct 03 '18 at 02:00
  • If you can tell me how the author arrives at this result of $-1/\sqrt{n}$, please do @Glen_b Otherwise I will keep reading. – Bucephalus Oct 03 '18 at 02:02
  • 1
    The problem with your algebra there isn't simply one of taking the square root of a negative number. You have stated $-k^2=-k$ (for a particular $k$), and irrespective of whether $k$ was positive, you don't get to manipulate powers like that and still claim to have equality. "$=$" isn't shorthand for "the next thing I thought about" -- it's a statement that two things are the same. $-k^2$ and $-k$ are not interchangeable quantities. – Glen_b Oct 03 '18 at 02:09
  • spread = $\sigma^2/n = \sigma/\sqrt{n}$, can $b = \sigma^2/n = \sigma/\sqrt{n}$ @Glen_b ? – Bucephalus Oct 03 '18 at 02:13
  • 1
    The same problem again (without the minus sign), how can you say "$\sigma^2/n = \sigma/\sqrt{n}$? What sort of thinking says a quantity is equal to its square? – Glen_b Oct 03 '18 at 02:15
  • I'm relatively new to statistics @Glen_b. Maybe there are others here who could answer why both of these are regarded as measures of spread.Just like $-\sigma^2/n$ and $-\sigma/\sqrt{n}$ might both be measures of bias. But I don't know....yet. – Bucephalus Oct 03 '18 at 02:16
  • 1
    This is not a statistics issue I'm pointing out -- it is a matter of very basic algebra; this is something that should be well in hand before you attempt to work with statistical issues. It would be a waste of time trying to deal with your question until you can understand the much more fundamental problem in your comprehension that I am raising here. Please take the time to try to carefully consider what I have been trying to explain. – Glen_b Oct 03 '18 at 02:20
  • 1
    I have considered that $2 \neq 3$ If you have constructive input to my question, that would be much appreciated. @Glen_b – Bucephalus Oct 03 '18 at 02:21

1 Answers1

2

The bias of $S_n^2$ as an estimate of variance is indeed $-\sigma^2/n$, as given in the text, & as you can confirm from e.g. Wikipedia. You probably have a early printing with a typo: the version currently on Google Books gives the code

c(varx, varx-1.0, -1/n) # sample variance and bias relative to true value 1.0 and expected value of bias

NB The terminology in the comment could be improved by substituting "error" for "bias": the difference between the estimate of the variance & its true value is the error & the expected value of the error its bias. (The "expected value of the bias" would just be the bias, as there are no random variables in the expression: $\operatorname{E}( -\sigma^2/n)=-\sigma^2/n$.)

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
  • That's very helpful @Scortchi. You're right, I just went and checked the book on safaribooks, and I have edited nto my post a screenshot of the code snippeted straight out of the book now. Good pickup. – Bucephalus Oct 03 '18 at 11:15