2

Let $T$ be some random variable on a probability space $\Omega$. Then we have, for $x\in\Omega$:

$$P(x) = P(x|T=T(x))P(T = T(x))$$

This equation is nonsense in an arbitrary probability space but informally it makes some kind of sense, and in a discrete space it's actually correct. Writing $g(t)=P(T=t)$ and $h(x)=P(x|T=T(x))$, we have

$$P(x)=g(T(x))h(x)$$

Which looks suspiciously like the Fisher-Neyman factorization theorem. If $T$ is sufficient for a family of distributions $P_\theta$, then we should instead write $g_\theta$ for $g$, but $h_\theta(x)=P_\theta(x|T=T(x))$ is indeed independent of $\theta$ by sufficiency, and the resemblance is even closer.

Can this interpretation be made rigorous, perhaps by thinking $g$ and $h$ as probability density functions?

Let $T$ be a random variable on some space $(\Omega, P)$ where $P$ is dominated by a measure $\mu$. Can the identity $$\frac{dP}{d\mu}(x) = P(x|T=T(x))P(T = T(x))$$ be made rigorous by replacing the meaningless probabilities on the right with appropriate pdfs?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Jack M
  • 369
  • 2
  • 8
  • It looks very much *not* like the F-N factorization theorem for the simple reason that $h$ is not intended to be a density function and often cannot even be normalized to one. Consider Normal distributions, for instance. – whuber Aug 13 '17 at 13:46
  • 1
    @whuber You're right, $h$ probably can't be thought of as a density function since the expression $P(x|T=T(x))$ doesn't even intuitively look like a density function. So some other type of object would have to be used to formalize $P(x|T=T(x))$. – Jack M Aug 13 '17 at 14:48

2 Answers2

1

Pollard argues that disintegrations, which give conditional probability distributions under fairly general conditions, are the right way to think about the factorisation theorem and sufficiency in the continuous case.

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
0

You could say that a statistic $T(x)$ is sufficient if you could model the sampling like first drawing the sufficient statistic from a distribution that depends on the parameters

$$T \sim f(t\vert \theta)$$

and then draw the observations $X$ from a distribution that depends only on $T$ and is independent from $\theta$

$$X \vert T \sim g(x \vert T)$$

In that case you have that the data $X$ does not tell anything more about $\theta$ than $T$.

And indeed for the distribution of X you get your expression where you multiply them. But, you would have to integrate it. To get a marginal distribution.

$$X\vert \theta \sim \int_{t \in \Omega} g(x\vert t) f(t \vert \theta) dt$$

Your expression is the joint distribution

$$h(x,t) = g(x\vert t) f(t \vert \theta)$$

For instance. To take a sample of size $n$ from a uniform distribution $U(0,a)$, you could first sample the maximum $t$ from a beta distribution and multiply with $a$, and then sample $n-1$ values from a uniform distribution $U(0,t)$.

The distribution of those $n-1$ variables tell nothing more about the value of $a$ than $t$.

Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
  • I guest that you might think why don't we have $h(x,t) = h(x)$ but I guest that this trick only works when we have $x$ as $n-1$ parameters. E.g in the case of the draw from the uniform distribution $t$ is the maximum from the sample and $x$ are the other $n-1$ values. If you would take $x$ as $n$ values then you get a degenerate function. For instance say you draw a samples of size two then $f(x_1,x_2 \vert t)$ will be infinite because it is all concentrated on a line instead in an area. https://stats.stackexchange.com/questions/361100/361541 – Sextus Empiricus Apr 01 '20 at 20:39