20

Is it enough to show that MSE = 0 as $n\rightarrow\infty$? I also read in my notes something about plim. How do I find plim and use it to show that the estimator is consistent?

Waldir Leoncio
  • 2,137
  • 6
  • 28
  • 42

1 Answers1

21

EDIT: Fixed minor mistakes.

Here's one way to do it:

An estimator of $\theta$ (let's call it $T_n$) is consistent if it converges in probability to $\theta$. Using your notation

$\mathrm{plim}_{n\rightarrow\infty}T_n = \theta $.

Convergence in probability, mathematically, means

$\lim\limits_{n\rightarrow\infty} P(|T_n - \theta|\geq \epsilon)= 0$ for all $\epsilon>0$.

The easiest way to show convergence in probability/consistency is to invoke Chebyshev's Inequality, which states:

$P((T_n - \theta)^2\geq \epsilon^2)\leq \frac{E(T_n - \theta)^2}{\epsilon^2}$.

Thus,

$P(|T_n - \theta|\geq \epsilon)=P((T_n - \theta)^2\geq \epsilon^2)\leq \frac{E(T_n - \theta)^2}{\epsilon^2}$.

And so you need to show that $E(T_n - \theta)^2$ goes to 0 as $n\rightarrow\infty$.

EDIT 2: The above requires that the estimator is at least asymptotically unbiased. As G. Jay Kerns points out, consider the estimator $T_n = \bar{X}_n+3$ (for estimating the mean $\mu$). $T_n$ is biased both for finite $n$ and asymptotically, and $\mathrm{Var}(T_n)=\mathrm{Var}(\bar{X}_n)\rightarrow 0$ as $n\rightarrow \infty$. However, $T_n$ is not a consistent estimator of $\mu$.

EDIT 3: See cardinal's points in the comments below.

  • 1
    @G.JayKerns Unbiasedness is unnecessary for this. Consider $S_n = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X_n})^2}$. $S_n$ is a biased estimator of the standard deviation yet you can use the above argument to show that it's consistent. –  Oct 29 '11 at 15:54
  • 1
    Looks good (+1); and I'll delete my earlier comments. –  Oct 29 '11 at 17:19
  • @G.JayKerns Your comments were a necessary addition. We must always make sure we're aware of the assumptions we're working under. –  Oct 29 '11 at 17:20
  • Just one more comment, and I'll go away. The numerator of the Chebychev fraction really should be $E(T_{n} - \theta)^{2}$, which does indeed match the variance when $E T_{n} = \theta$. Writing it that way, though, makes it clear that Beatrice was right with the first sentence of her question, because, of course, that's the definition of MSE. –  Oct 29 '11 at 17:30
  • Good point. I just realized I completely ignored her first sentence. –  Oct 29 '11 at 17:37
  • 3
    @MikeWierzbicki: I think we need to be very careful, in particular with what we mean by *asymptotically unbiased*. There are at least two different concepts that often receive this name and it's important to distinguish them. Note that it is *not true* in general that a consistent estimator is asymptotically unbiased in the sense that $\mathbb E T_n \to \theta$ even when the mean $\theta_n = \mathbb E T_n$ exists for all $n$. Many people call the convergence $\mathbb E T_n \to \theta$ *unbiasedness in the limit* or *approximate unbiasedness*...(cont.) – cardinal Oct 29 '11 at 18:56
  • 1
    Obviously, in order for a consistent estimator to be biased in the limit, the convergence in $L_2$ must fail since $\mathbb E(T_n - \theta)^2 = \mathrm{Var}(T_n) + (\theta_n - \theta)^2$ where $\theta_n = \mathbb E T_n$. – cardinal Oct 29 '11 at 18:57
  • @G.JayKerns: Please consider letting your comments remain in the future. I find them enlightening and sometimes they can be helpful for me (and others?) when reading the comment stream, even if the comments don't appear immediately relevant to an answer that has been edited to reflect the comments. :) – cardinal Oct 29 '11 at 19:04
  • @cardinal Great comments - I was sloppy in one of my comments. A consistent estimator need not be asymptotically unbiased -- I realize one of my comments from before may say otherwise, I think I shall delete it. To summarize my response succinctly: if you can show that $T_n$ is asymptotically unbiased and $E(T_n - \theta)^2\rightarrow 0$ then $T_n$ is consistent. However this is not the only way in which consistent estimators arise, nor is it the only way to prove consistency. Is that where we stand in the discussion as of now? –  Oct 29 '11 at 19:14
  • @MikeWierzbicki: Thanks for the response. Note that you don't even need the "$T_n$ is asymptotically unbiased" part of your previous comment since $\mathbb E(T_n - \theta)^2 \to \theta$ is already enough. :) And, yes, this is certainly not the only way for consistency to arise, though it's one of the more common, as you previously noted. One easy way to convince yourself of the fact that $L_2$ convergence is not the only way is that the concept of convergence in probability would not be necessary otherwise! :) – cardinal Oct 29 '11 at 19:19
  • @cardinal, I think there is a typo in your comment since I'm pretty sure $E(T_n - \theta)^2 \rightarrow \theta$ is not enough ;) – Macro Jun 25 '12 at 13:16
  • @Macro: Wow. That one had escaped me even after rereading that a few times. This particular typo (and its complement) is actually quite common in older statistical texts (and a few newer ones); undoubtedly it is usually a transcription error due to a combination of poor handwriting and/or the transcriptionist not being fully versed on the material. I wish I could claim that excuse in this case! – cardinal Jun 25 '12 at 13:27