Conditioning in the definition of sufficient statistics

Question

Let $X_1,...,X_n$ be an i.i.d. sample with parameter $\theta$ and $T$ a statistics. The statistics is called sufficient if, given a value $t$, the distribution $P_{\theta}(X_1,..,X_n|T=t)$ does not depend on $\theta$.

But what is the range of $t$ ? Do we need also that the event $P_{\theta}(T=t)>0$ for every value of $\theta$ in order to be sure that we are not conditioning on null events ? Do we need that $Supp_{\theta} f_T$ is the same for every value of $\theta$ ? Does it make sense this doubt ?

ps: the question is posed in the setting of discrete distribution.

Answer 1

There is no impact on sufficiency of the fact that $\mathbb{P}_\theta(T=t)$ is not always positive. As indicated in my example, the statistic $$T=X_{(n)}=\max_{1\le i\le n} X_i$$ is sufficient when the $X_i$'s are uniform on $\{1,\ldots,\theta\}$. And $$\mathbb{P}_\theta(T=t)=0$$ when $\theta<t$. The joint probability mass function of $(X_1,\ldots,X_n,X_{(n)})$ at $(x_1,\ldots,x_n,x_{(n)})$ is also zero when $x_{(n)}>\theta$, hence the conditional probability of $(X_1,\ldots,X_n)$ given $X_{(n)}=t$ and $t>\theta$ is not defined. But since the conditional distribution of $(x_1,\ldots,x_n)$ given $X_{(n)}=t$ is uniform over $$\{(x_1,\ldots,);\ x_{(n)}=t\}$$ independently of $\theta\ge t$ (and not defined otherwise), this does not impact sufficiency.

I presume that the undefined nature of the conditional in the impossible situation that $X_{(n)}=t$ and $t>\theta$ appears to bring some dependence on $\theta$ but this is not a correct impression: the conditional is not defined because the conditioning event is impossible. The part that brings information on $\theta$ is $X_{(n)}$, which is fine since it is sufficient.

Here is a quote from one of the earlier papers on the topic, by Koopman (1935):

where he similarly sees no impact on sufficiency in the fact that the density may be null for the actual observations and some values of the parameter.

Answer 2

In advanced probability texts, sufficiency is usually defined formally in terms of partitions on the sample space, and then we build up the standard definition as an implication of this for a parametric model. In any case, once you translate to the common definition, the requirement for sufficiency is that this condition should hold for all $t$. So a statistic $T: \mathbb{R} \rightarrow \Lambda$ will be sufficient for $\theta$ if and only if it has a conditional probability function $P_{\theta}(\mathbf{X}|T =t)$ satisfying:

$$P_{\theta}(\mathbf{X}|T=t) = P_{\theta'}(\mathbf{X}|T=t) \quad \quad \quad \text{for all } \theta, \theta' \in \Theta \text{ and } t \in \Lambda.$$

Note that in cases where $\mathbb{P}_\theta(T=t)=0$ for some $t$, the conditional probability is defined through the law of total probability --- i.e., it is any measureable non-negative function satisfying:

$$\int \limits_\Lambda P_{\theta}(\mathbf{X} \in \mathcal{S}|T=t) dF_T(t) = \mathbb{P}_{\theta}(\mathbf{X} \in \mathcal{S}) \quad \quad \quad \text{for all } \theta \text{ and measureable } \mathcal{S}.$$

This latter part gives you some "wiggle room" for sufficiency in regard to sets with probability measure zero. If there is any conditional probability function satisfying the above then we would say that the statistic $T$ is sufficient.

---

If you would like to know more about the underlying definition, I recommend you have a look at the definition of a sufficient partition first. Most textbooks on intermediate or advanced probability will have an explanation of sufficiency that is couched in an initial definition in terms of a partition on the sample space.