4

I came cross an interesting comment saying

If the dimension of a sufficient statistic $T(X)$ is the same as that of the parameter space, then $T(X)$ is minimal sufficient.

Is this is true? I examined some examples e.g. full-rank exponential families, curved exponential families e.g. $N(\theta, \theta^2)$, and some uniform distributions, e.g. $U(-\theta,\theta)$, $U(0,\theta)$, $U(\theta,\theta+a)$ and found their sufficient statistic has the same dimension with the parameter space and indeed they are minimal sufficient.

Is this quoted statement correct? Can you please provide a reference of a proof, or a counter example?

Tan
  • 1,349
  • 1
  • 13
  • 3
    Hint: for a Normal$(\mu,\sigma^2)$ family, let $\bar x$ and $s^2$ be the sample mean and variance, respectively, and let $x_1$ be the first observation in sample. It's well-known that $(\bar x, \sqrt{s^2})$ is minimal sufficient. Define $T=(\bar x, \operatorname{sgn}(x_1)\sqrt{s^2}).$ (1) What is the dimension of $T$? (2) Is $T$ minimal sufficient? – whuber Jan 07 '21 at 19:08
  • 2
    @whuber (1) 2; (2) No. $T$ is a two-dimensional sufficient statistic for the two-dimensional parameter $(\mu,\sigma^2)$. But $T$ is not minimal sufficient statistic as $(\bar{x},\text{sgn}(x_1)\sqrt{s^2})$ and $(\bar{x},\sqrt{s^2})$ is not one-to-one corresponded. – Tan Jan 07 '21 at 19:19
  • How come this is valid for $N(\theta,\theta^2)$ if you take a sample of size $n(>1)$? Another counterexample is the $U(\theta,\theta+1)$ family. – StubbornAtom Jan 07 '21 at 19:54
  • 1
    @StubbornAtom for $N(\theta,\theta^2)$ sufficient statistic $(\sum X_i, \sum X_i^2)$ is two dimensional and $(\theta,\theta^2)$ lives in $\mathbb{R}^2$, and it happens that $(\sum X_i, \sum X_i^2)$ is also minimal sufficient statistic. So it satisfied the quoted statement. For $U(\theta,\theta+1)$, sufficient statistic is two-dim $(X_{(1)},X_{(n)})$ but $\theta$ is one-dim. Thus $U(\theta,\theta+1)$ is not a suitable example for the quoted statement in OP. – Tan Jan 07 '21 at 20:05
  • 1
    Okay. But then $N(\theta,\theta^2)$ is not suitable either as $\theta$ is one-dimensional. – StubbornAtom Jan 07 '21 at 20:27
  • @StubbornAtom Actually, I believe $(\theta,\theta^2)$ is two-dimensional because $(\theta,\theta^2)\subset \mathbb{R}^2$. Also, it belongs to the curved exponential family, which means the parameter space does not contain an open set in $\mathbb{R}^2$. – Tan Jan 07 '21 at 20:43
  • 2
    @Tan: In the $N(θ,θ^2)$ model, the parameter is one-dimensional even though this is a two-dimension (curved) exponential family. – Xi'an Jan 08 '21 at 08:33
  • @Xi'an Sorry for the confusion. By two-dimensional, I actually mean two-dimensional parameter $\textbf{space}$. For example in the minimal exponential family case, it is the $d$, which is the size of the natural parameter. So for $N(\theta,\theta^2)$ is 2. Plz correct me if I'm wrong. – Tan Jan 08 '21 at 17:45
  • 2
    @Tan what is the parameter space of $N(\theta,\theta^2)$? Don't you have a single parameter $\theta \subset \mathbb{R}^1$? – Sextus Empiricus Feb 03 '21 at 14:42

2 Answers2

2

The construction by Whuber in the comments gives an example to create a sufficient statistic that is not minimal sufficient.

The trick of that counterexample is to design a sufficient statistic that has a larger space, but still the same dimension. For a normal distribution with fixed $\mu$ and parameterized by only $\sigma$, the parameter is the set of positive numbers $\sigma \subset \mathbb{R_{>0}}$ and the minimal sufficient statistic has the same space $s \subset \mathbb{R_{>0}}$. This leaves room for the creation of a statistic $s^\prime \subset \mathbb{R}$ by adding the negative numbers (in the example by Whuber this is done my multiplying with the sign of a variable $x_1$), which has a non-injective surjective relationship with the minimal sufficient statistic $s$ such that the inverse function does not exist.

d

Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
  • I need to improve this answer. After one year re-reading it I had troubles understanding it. I have some ideas to do these improvements but am not gonna do it right away. If anybody reads this comment after a month (Feb 2022 or further) then I have forgotten it and please remind me to edit the question. – Sextus Empiricus Jan 12 '22 at 17:58
1

The quoted statement is incorrect. The post Minimal sufficient statistic whose dimension is less than dimension of parameter has one example, @whuber another one in a comment.

This: Minimum dimension of sufficient statistics is also relevant.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • The first link's sufficient statistic's dimension does not equal to the parameter space dimension. Thus it is not a counterexample of the statement. – Tan Jan 29 '21 at 17:43
  • @Tan: Did you look at the answer in that post? It is an example. – kjetil b halvorsen Jan 29 '21 at 17:46
  • I think so.. It says when sample size is one then sufficient statistic $X$ is a minimal sufficient statistic for $\text{Beta}(\alpha,\beta)$. So $X$ is one dimensional and $(\alpha,\beta)$ is two dimensional. My question was asking when sufficient statistic dimension is same (not less) with parameter space dimension, would the sufficient be minimal? – Tan Jan 29 '21 at 22:50