1

This is more a conceptual question, but it seems to me that a sufficient statistic for a parameter is a concepts that applies only if we want to estimate the parameter via maximum likelihood. Is this right or wrong?

People usually claim that sufficient statistics are a data reduction technique. That is, I can store only the summary data expressed by the form of the observed value of the sufficient statistic and I will have enough information to make inference about the parameter of interest, $\theta$, in the future. To me, this is true only if we will make inference via maximum likelihood.

To expand, I might be able to come up with a better estimator for $\theta$ which takes on a strange functional form, and maybe then the observed value of the sufficient statistic will not be enough to allow me to infer about $\theta$ in the way that I wish.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
DanRoDuq
  • 506
  • 3
  • 12
  • 1
    We ought first to establish that this question actually has some content. Could you provide us an example of any reasonable statistical procedure that requires more information than the likelihood? – whuber Jan 26 '18 at 18:54
  • well, if I think of the method of moments estimator, there is no guarantee that the resulting method of moments estimator will be a function of the sufficient statistic, is there? – DanRoDuq Jan 26 '18 at 19:30
  • 3
    You might want to look at the Rao-Blackwell Theorem https://en.wikipedia.org/wiki/Rao–Blackwell_theorem for more information about the usefulness of sufficient statistics. – jbowman Jan 26 '18 at 19:33
  • 1
    Hmm interesting, so given an estimator g(x), obtained whichever way I want, then I can construct one that is a function of a sufficient statistic via E[g(x)|T(x)] and it will be no worse in terms of the expected square distance. This is nice, but now we have had to restrict what we mean be a "good estimator" which is nowhere considered when talking about sufficient statistics. – DanRoDuq Jan 26 '18 at 19:45
  • 1
    That's a fair criticism. It gets to the heart of the problem many people have with maximum likelihood in the first place: it doesn't account *at all* for any loss function. However, expected squared loss enjoys some rather general properties. Furthermore, anyone using MM estimators isn't considering a loss function either, so the same criticism can (and should be) be leveled at them. – whuber Jan 26 '18 at 21:23

1 Answers1

0

It is not true that sufficient statistics are only relevant with maximum likelihood estimators. For instance, knowing a sufficient statistic is enough for Bayesian inference, see the calculation in Reducing dimension $P(\theta|y) = P(\theta|s)$ in the posterior distribution.

People usually claim that sufficient statistics are a data reduction technique. That is, I can store only the summary data expressed by the form of the observed value of the sufficient statistic and I will have enough information to make inference about the parameter of interest, , in the future. To me, this is true only if we will make inference via maximum likelihood.

See the Rao-Blackwell theorem: Understanding the Rao-Blackwell Theorem and at Wikipedia.

Some other relevant posts:

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467