15

The definition for completeness is that if a statistic $s(x)$ is complete, we have that for every measurable $g$, $$E_\theta(g(s(x))) = 0\,, \ \forall\,\theta\ \Rightarrow\ g(s) = 0 \text{ a.s.}$$

I've heard that we can think of completeness as saying that if we wanted to estimate the zero function using a complete $s(x)$, among the class of all zero-unbiased functions of the statistic, the only one is one which takes value 0 almost surely. This seems like a bizarre notion--why would we want to estimate the zero function?

I've also heard that in estimating parameters of a probability model $P_\theta$, one does not need any more than a sufficient statistic. I heard that having any more than the sufficient statistic provides no additional information. How does this connect to the definition of completeness above (Basu's, maybe?) ?

Is there some better intuition for the (seemingly) bizarre condition above?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Christopher Aden
  • 1,775
  • 4
  • 24
  • 43
  • See also my relevant answer [here](https://stats.stackexchange.com/questions/196601/what-is-the-intuition-behind-defining-completeness-in-a-statistic-as-being-impos) – kjetil b halvorsen Feb 07 '19 at 10:05

2 Answers2

14

By virtue of the Rao-Blackwell theorem, if $\delta(x)$ is an unbiased estimator of $h(\theta)$ and $S$ is a sufficient statistic, then

$$ \mathbb{E}[\delta(X)|S(X)=s] = \delta^*(s) $$

is a "better" estimator in the sense that it has a smaller variance while being still unbiased. In this restricted sense, using only a sufficient statistic makes sense. (However, the remainder of the data can help in estimating the precision of this estimator and so cannot be discarded.)

When $S$ is in addition complete, there can only be one unbiased estimator based on $S$, as pointed out by phaneron. Following from the Lehmann–Scheffé theorem, this is the best unbiased estimator.

Community
  • 1
Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • 2
    Thanks for the answer! Why does completeness ["ensures that the parameters of the probability distribution representing the model can all be estimated on the basis of the statistic: it ensures that the distributions corresponding to different values of the parameters are distinct"](http://en.wikipedia.org/wiki/Completeness_(statistics))? please also see my question here http://stats.stackexchange.com/q/53107/1005. – Tim Sep 04 '14 at 13:38
13

For completeness

Recall if $E(f(x)) = u$ then $f(x)$ is an unbiased estimate of $u$.

Now so will $g(x) = f(x) + h(x)$ if $E(h(x)) = 0$

So its to assure that $f(x)$ is the unique unbiased estimate. (The rest and many fine details I have forgotten.)

For sufficiency

The easiest way to appreciate sufficiency is via the relative belief ratio which is just the posterior probability divided by the prior probabilty (this is k * likelihood) calculated via ABC or two stage simulation. After conditioning on a sufficent statistic, conditioning on anything further does not change the posterior distribution. If two different statistics are sufficient, conditioning on either will give the same posterior distribution and the minimal sufficent statistic will give the best approximation of the same posterior distribution. But if this is for a course this will likely just be distracting as you will likely need to answer in terms of that statistic indexing the likelihood function.

phaneron
  • 1,252
  • 7
  • 7