Trying to make sense of claims regarding Rao-Blackwell and Lehmann-Scheffé for sufficient/complete statistics

Question

I am currently trying to learn the two related concepts of the Rao-Blackwell theorem and the Lehmann-Scheffé theorem.

Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2 < \infty$. We have that $E[S^2] = \sigma^2$, where $S^2 = \sum_{i = 1}^n \dfrac{(X_i - \bar{X})^2}{n - 1}$ and $\bar{X} = \sum_{i = 1}^n \dfrac{X_i}{n}$. Now assume the $X_i$ are Poisson random variables with parameter $\lambda$. My understanding is that, using Lehmann-Scheffé, we get that $E[S^2 \mid \bar{X}] = \bar{X}$. Then, using the law of total variance, we get that $\text{Var}(S^2) > \text{Var}(\bar{X})$. Based on what I've read, the two above theorems imply that, if we define, say, a sufficient statistic $T_1(\mathbf{X})$ and a complete sufficient statistic $T_2(\mathbf{X})$ for some parameter $\varphi$, then, under some conditions, we can say that $\text{Var}(T_1(\mathbf{X})) > \text{Var}(T_2(\mathbf{X}))$. However, I'm having trouble understanding this last part. I've read over various notes on the subject, but I'm still not sure I understand what it's saying. Why can we say that $\text{Var}(T_1(\mathbf{X})) > \text{Var}(T_2(\mathbf{X}))$? And what are these 'conditions' that make this inequality valid?

Answer 1

There is a complete sufficient statistic for $\theta$ in a model ${\cal P}_\theta$ if and only if the minimal sufficient statistic is complete (according to Lehmann "An Interpretation of Completeness and Basu’s Theorem"). This means you can't have distinct $T_1(X)$ and $T_2(X)$ the way you want. As the paper says (first complete paragraph of the second column)

On the other hand, existence of a complete sufficient statistic is equivalent to the completeness of the minimal sufficient statistic, and hence is a property of the model ${\cal P}$.

That is, in any given model, if $T_2$ is complete sufficient and $T_1$ is sufficient, $T_1$ is also complete sufficient.

The two theorems say

1/ Rao-Blackwell: Conditioning on any sufficient statistic will reduce the variance. This follows from the law of total variance

2/ Lehmann-Scheffé: In the special case that the model has a complete sufficient statistic, you get a fully efficient estimator.

In the Poisson case, the minimal sufficient statistic $\bar X$ is complete and the two estimators are identical.

There's an interesting example here of a situation where a Rao-Blackwell-type estimator is not fully efficient (not even admissible). The model is $X\sim U[\theta(1-k),\theta(1+k)]$ for known $k$ and unknown $\theta$. The Cramér-Rao bound does not apply, since the range of $X$ depends on $\theta$. A sufficient statistic is the pair $(\min X_1, \max X_n)$, and any observation is an unbiased estimator, however $E[X_1|(\min X_1, \max X_n)]$ is not even the best linear function of the two components of the sufficient statistic.

A couple more points to fill in potential gaps:

1. This leaves open is whether there might be an unbiased estimator attaining the Cramér-Rao bound that isn't obtainable using the Lehmann-Scheffé theorem. There isn't (in reasonably nice models): any model where the bound is attained has a score function of the form $$\frac{\partial \ell}{\partial \theta}=I(\theta)(g(x)-\theta)$$ for some $g()$ (where $I()$ is the information), in which case $g(x)$ is a both a complete sufficient statistic for $\theta$ and the minimum variance unbiased estimator.

2. As @AdamO indicates, none of this translates tidily to asymptotics: there are asymptotically unbiased estimators that beat the asymptotic information bound at a point (Hodges superefficient estimator) and even on a dense set of measure zero(Le Cam's extension of Hodge's estimator). The best you can do is the Local Asymptotic Minimax theorem, which says you can't beat an 'efficient' estimator uniformly over neighbourhoods of $\theta_0$ with diameter $O(n^{-1/2})$