12

My understanding of what an estimator and an estimate is: Estimator: A rule to calculate an estimate Estimate: The value calculated from a set of data based on the estimator

Between these two terms, if I am asked to point out the random variable, I would say the estimate is the random variable since it's value will change randomly based on the samples in the dataset. But the answer I was given is that the Estimator is the random variable and the estimate is not a random variable. Why is that ?

Xi'an
  • 90,397
  • 9
  • 157
  • 575
Kanmani
  • 287
  • 2
  • 14

2 Answers2

25

Somewhat loosely -- I have a coin in front of me. The value of the next toss of the coin (let's take {Head=1, Tail=0} say) is a random variable.

It has some probability of taking the value $1$ ($\frac12$ if the experiment is "fair").

But once I have tossed it and observed the outcome, it's an observation, and that observation doesn't vary, I know what it is.

Consider now I will toss the coin twice ($X_1, X_2$). Both of these are random variables and so is their sum (the total number of heads in two tosses). So is their average (the proportion of head in two tosses) and their difference, and so forth.

That is, functions of random variables are in turn random variables.

So an estimator -- which is a function of random variables -- is itself a random variable.

But once you observe that random variable -- like when you observe a coin toss or any other random variable -- the observed value is just a number. It doesn't vary -- you know what it is. So an estimate -- the value you have calculated based on a sample is an observation on a random variable (the estimator) rather than a random variable itself.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • 1
    +1, the thread worth mentioning is:https://stats.stackexchange.com/questions/7581/what-is-the-relation-between-estimator-and-estimate – Tim Dec 07 '17 at 09:58
  • 1
    but once we observe, why is it an estimate at all? there is nothing to be estimated after observation? – Parthiban Rajendran Nov 18 '18 at 16:44
  • 3
    It's an estimate of an unobserved population parameter. For example in the coin tossing experiment where you don't know the coin to be fair, the observed average number of heads in $n$ tosses is a suitable estimate of the probability of a head. – Glen_b Feb 15 '19 at 00:06
  • I'm really confused now because @Tim linked a thread that explicitly said an estimator is not a random variable – Colin Hicks Mar 11 '19 at 21:15
  • If you have a function (say with vector argument), $g$, then $g$ is just a function, but the value of that function when $g$ is applied to a collection of variates ($X=(X_1,X_2,...,X_n)$) whose components are random variables (peerhaps corresponding to some random sampling procedure on some population), then $T=g(X)$ will be a random variable. If you were to define $g$ as the estimator then $g$ is just a function. But if you called $T$ the estimator then $T$ is a random variable. Strictly this latter usage (as I have above) is rather loose (but quite common). ... ctd – Glen_b Mar 11 '19 at 22:10
  • ctd... Consider the [Wikipedia article](https://en.wikipedia.org/wiki/Estimator) -- it starts by calling an estimator a 'rule' (which it subsequently calls $\widehat{\theta}$, like "$g$" above) but then later says "Being a function of the data, the estimator is itself a random variable" (as I have there), then shifts back to talking about "applying it to a random variable"; going back to being consistent with whuber's discussion. It then says "Often an abbreviated notation is used in which $\widehat{\theta}$ is interpreted directly as a random variable, but this can cause confusion." ...ctd – Glen_b Mar 11 '19 at 22:24
  • ctd... (which indeed the article itself had done as just noted above). ... To be clear -- strictly, whuber is right, but the conflation of $T$ with $g$ is common. – Glen_b Mar 11 '19 at 22:30
  • I think it's common to treat an r.v. symbol such as $X$ both as the r.v. per se and as its value, depending on context. Some authors are very careful to avoid this ambiguity in order to avoid confusion among the unsophisticated (e.g. me). – Mars May 12 '19 at 03:19
1

My understandings:

  1. An estimator is not only a function, which input is some random variable and output another random variable, but also a random variable, which is just the output of the function. Something like $y=y(x)$, when we talk about $y$, we mean both the function $y()$, and the result $y$.
  2. Example:an estimator $\overline X=\mu(X_1,X_2,X_3)=\frac{X_1+X_2+X_3}{3}$, we mean both $\mu()$ ,which is a function,and its result $\overline X$,which is random variable.
  3. The difference between estimator and estimate is about before observing or after observing.
  4. Actually, similar to an estimator, an estimate is both a function and a value(the function output) too. But the estimate is in the context of after observing, and by contrast, the estimator is in the context of before observing.

A picture illustrates the idea above:enter image description here

I have researched this question during my weekend, after reading lots of material from the internet, i am still confused. Although I am not completely sure that my answer is right, it seems like, to me, it's the only way to let everything make sense.

dawen
  • 131
  • 5
  • 2
    +1 You are making some good distinctions. Given your interest and dedication, might I recommend consulting a good textbook rather than relying entirely on the Internet? Textbooks can go deeply into a subject in a consistent manner, whereas depth and consistency are very hard to find online. – whuber May 12 '19 at 16:11
  • 1
    hi whuber, I highly recommend this https://newonlinecourses.science.psu.edu/stat414/ as an undergraduate level learning material of probability and statistic, and All of Statistics by Larry is also a good book for the beginner. Almost all my stat teachers recommend mathematical statistics by j. shao as a graduate level Textbook. I do agree with you that consistency and depth are very important for learning, I think textbooks and courses are for consistency while wiki and StackExchange are for depth. – dawen May 13 '19 at 10:13