Is it true that an estimator will always asymptotically be consistent if it is biased in finite samples? I feel like it is true but not sure exactly how to prove that...
Asked
Active
Viewed 960 times
11
-
10Suppose this were so. Then take any such estimator and add some arbitrary constant to it so that it remains biased in finite samples: by construction it is *not* asymptotically consistent. – whuber Dec 10 '20 at 14:44
-
@whuber, this should be an answer, and it would arguably be the best among the ones posted. – Richard Hardy Dec 11 '20 at 08:48
-
@RichardHardy Not the whole story. Asymptotic consistency is useless without evidence of sufficient convergence. – Carl Dec 11 '20 at 09:45
-
2@Carl, this is beyond the OP's question. What do you mean by *sufficient convergence*? – Richard Hardy Dec 11 '20 at 09:48
-
@RichardHardy Sufficient for confidence in the results, as in confidence intervals, without which one is guessing as to validity. I took it upon myself to answer in the useful context, and not just lay down alms on the altar of asymptotic consistency. So, shoot me. – Carl Dec 11 '20 at 09:55
-
@RichardHardy Sometimes reducing variance is more important than asymptotic consistency, as in Tikhonov regularization, so, a more complete answer is arguably more important than a yes/no one. Indeed the answer to the OP question is a simple "NO" so how can you justify any answer longer than that? And, a 'No" answer is not even long enough for a comment on this site. The devil is in the details, and it is a matter of taste as to how much of the issue one characterizes. Your preference on this is more narrow than mine, congrats. – Carl Dec 11 '20 at 10:10
-
A similar question is discussed in [Wikipedia](https://en.wikipedia.org/wiki/Consistent_estimator#Bias_versus_consistency). – Richard Hardy Dec 11 '20 at 15:01
-
*"I feel like it is true"* I wonder about the intuition behind this. – Sextus Empiricus Dec 11 '20 at 15:12
4 Answers
22
Consider the estimator $\hat{\theta} = 3$. If this estimator is estimating a parameter that is not equal to three then it is biased in all finite samples. Is this estimator asymptotically consistent?
Ben
- 91,027
- 3
- 150
- 376
5
Next to the simple and effective example of Ben, here is a more applied and specific one: Imagine you try to estimate a causal effect, but your regression model to estimate this causal parameter is misspecified (often called "omitted variable bias").
A well-known example deals with returns to schooling, i.e., how much more you earn due to (i.e., in the sense of a causal effect of sitting in class, taking courses) additional schooling. If you simply regress earnings on schooling the regression is likely misspecified, because people (after compulsory schooling ends) choose how much schooling they want. Now, more motivated and able students will, as a tendency, find the idea of going to school for longer less daunting than other students. Now, such able and motivated persons will however also be likely to be good at the workplace due to these characteristics, irrespective of how much schooling they have. Hence, they will likely earn more.
Hence, you would need to control for things like ability/motivation - which may not be easy in practice - in your regression (and likely other things to).
Just collecting more data on your simple regression of earnings on schooling will, in turn, not save you from this problem, so both biased and inconsistent estimation. For both small and large datasets, the simple regression, as a tendency, compares earnings of students who are both able and have higher schooling to earnings of students who are less able and have less schooling. Assigning the entire difference in earnings to schooling hence will overstate the causal effect of schooling.
Christoph Hanck
- 25,948
- 3
- 57
- 106
2
Yes, in some circumstances, no in others. For example, if a bias results from a self-inconsistent assumption, then no. Examples of this latter include omitted variable bias and AIC in the case of censored data, which violates the maximum likelihood assumption. Examples of when it pertains would be AIC in the case of complete support (i.e., without censoring such that the maximum likelihood assumption pertains), and ordinary least squares for equidistant independent axis data. In still others, for example, variance is generally unbiased, but standard deviation is not, see this. Standard deviation would still be asymptotically correct because the small number bias would reduce to zero for $n\to\infty$. Nevertheless, one should not rely on just any asymptotic convergence, if a rather better estimator is available, and see how this was done in this example. Briefly, if you small number correct standard deviations from a large number of 2 sample SD's and then average them, you will obtain a more variable estimate than if you root mean square combine all the variances and then use a much lesser small number correction for the total number of trials. Some people are surprised at how ineffectual AIC can be for small samples. Thus, how fast asymptotic convergence occurs can be critical to interpretation of statistical results, and sometimes, for example for AIC, when we do not have measures that inform us of how precise or accurate statistical results are, it can be problematic.
Thus, the question of whether or not a procedure is asymptotically convergent is not by itself a sufficient criterion of validity of statistical results. We also need confidence intervals for those results.
Carl
- 11,532
- 7
- 45
- 102
-
1
-
-
-
1@Ben Ha, ha. Glad I could tickle your funny-bone. At least someone, you, or should I call you "them" has a sense of humor on this depressing site. – Carl Dec 11 '20 at 09:39
-
1
-
1@Ben You are a joy, and put lightness in my heart. No, I was making fun of the web site, and "their" misplaced sense of propriety. – Carl Dec 11 '20 at 10:01
0
Is it true that an estimator will always asymptotically be consistent if it is biased in finite samples?
The correct reply is trivial and it is NO, as pointed out above.
However your question immediately suggest a more interesting one:
Is it true that an estimator will always asymptotically be consistent if it is unbiased in finite samples?
The reply is: yes, if its variance going to zero when sample size diverge.
I add this part here because I suppose can be interesting for some readers.
markowitz
- 3,964
- 1
- 13
- 28
-
4What if the variance is decreasing but converges to a positive number rather than zero? – Richard Hardy Dec 11 '20 at 10:18
-
I Richard! In my experience I never encounter situation like this. However I suppose that no kind of consistency is achieved if the estimator variance have a lower bound greater that zero. – markowitz Dec 11 '20 at 13:23
-
-
-
Though what if the variance is infite but, say, the mean absolute deviation goes to zero? – Richard Hardy Dec 11 '20 at 15:00
-
Estimators with infinite variance is an advanced topic. I think that for address this point, first of all, we should be agree on all details about the meaning of consistency. Here I just wanted to give an easy intuition to the readers. – markowitz Dec 11 '20 at 16:18
-
Sure, and that is great. I just think the devil is in the details, so better get them right. – Richard Hardy Dec 11 '20 at 19:09