Statistics can't be a function of a parameter - but isn't the sample a function of the parameter?

Question

I have a question that relates to this post:

Can a statistic depend on a parameter?

But on it, the discussion focuses much on the t-statistic given as an example by the question asker. My doubt in a broader sense is that:

Let ${X_1, ..., X_n}$ be a random sample of size $n$ from a population. $T(x_1, ..., x_n)$ is a real-valued function. The random-variable $Y = T(X_1, ..., X_n)$ is called a statistic.

The statistic can't be a function of any parameter. But the random sample ${X_1, ..., X_n}$ depends on some parameter $\theta$. So, if the statistic is a function of the random sample, and the random sample is a function of a parameter, doesn't that make the (random) statistics a function of the parameter as well?

I understand that when we are calculating a t-statistic, say, we aren't using the real parameter of the population anywhere. But we're using a sample mean. And this sample mean is dependent on the populational mean, ain't it? So the (random) statistic depends in some sense of the populational mean.

Then, $T(\textbf{X}) = T(\textbf{X}(\theta))$. But that goes against the fact that the statistic can't be a function of any parameter. That just doesn't enter my head when I think of the random counterpart of the statistic.

There must be something wrong with my line of thought but I just can't find it. Any thoughts?

Answer 1

$\require{mediawiki-texvc}$Let $T=T(X)=T(X_1,X_2, \dotsc, X_n)$ be a statistic, and assume we have some statistical model for the random variable $X$ (the data), say that $X$ is distributed according to the distribution $f(x;\theta)$, $f$ is then a model function (often a density or probability mass function) which is known only up to the parameter $\theta$, which is unknown.

Then the statistic $T$ has a distribution which depend upon the unknown parameter $\theta$, but $T$, as a function of the data $X$, do not depend upon $\theta$. That only says that you can calculate the realized value of $T$, from some observed data, without knowing the value of the parameter $\theta$. That is good, because you do not know $\theta$, so if you needed $\theta$ to calculate $T$, you would not be able to calculate $T$. That would be bad, because you could not even start your statistical analysis!

But, still the distribution of $T$ depends upon the value of $\theta$. That is good, because it means that observing the realized value of $T$ you can guess something about $\theta$, maybe calculate a confidence interval for $\theta$. If the distribution of $T$ was the same for all possible values of $\theta^\P$, then observing the value of $T$ would not teach us anything about $\theta$!

So, this boils down to: You must distinguish between $T$ as a function of the data, and the distribution of the random variable $T(X)$. The first one do not depend upon $\theta$, the second one does.

$\P$: Such a statistic is called ancillary. It might be useful, just not directly, alone for inference about $\theta$.

Answer 2

The confusion here stems from conflating a random variable with its distribution. To be clear about the issue, a random variable is not a function of the model parameters, but its distribution is.

Taking things back to their foundations, you have some probability space that consists of a sample space $\Omega$, a class of subsets on that space, and a class of probability measures $\mathbb{P}_\theta$ indexed by a model parameter $\theta$. Now, the random variable $X: \Omega \rightarrow \mathbb{R}$ is just a mapping defined on the domain $\Omega$. The random variable itself does not depend in any way on the parameter $\theta$, and so it is wrong to write it as a function $X(\theta)$. It is of course true that the probability distribution of $X$ depends on $\theta$, since the latter affects the probability measure over the sample space. However, it does not affect the sample space itself.$\dagger$

Consequently, when you are dealing with a statistic, which is just a function of the observed random variables, this also does not depend on $\theta$, but its distribution usually does. (If not, it is an ancillary statistic,)

---

$\dagger$ This treatment has taken $\theta$ as an index for the probability measure, but the same result occurs under a Bayesian treatment where $\theta$ is regarded as a random variable on $\Omega$ and the behaviour of $X$ is treated conditionally on the parameter.

Answer 3

While the other answers (so far) are quite to the point and valid, I would like to add another direction to the discussion that relates to both fiducial inference (Fisher's pet theory) and a form of sampling called "perfect sampling" (or "sampling from the past").

Since a random variable is a measurable function from a (probability) space $(\Omega,\mathbb{P})$ to $\mathbb{R}$ (or $\mathbb{R}^n$), $X:\Omega\to\mathbb{R}$, the function may itself depend on a parameter $\theta$ if its distribution depends on $\theta$, in the sense that $X=\Psi(\omega,\theta)$. For instance, if $F_\theta$ denotes the cdf of $X$, we can write $X=F_\theta^{-1}(U)$ where $U$ is a uniform $\mathcal{U}(0,1)$ random variable. In this sense, $X$ (and the sample $(X_1,\ldots,X_n)$ as well) can be written as a [known] function of the [unknown] parameter $\theta$ and a fixed distribution [unobserved] random vector $\xi$, $$(X_1,\ldots,X_n)=\Psi(\xi,\theta)$$ This representation is eminently useful for simulation, either for the production of (pseudo-) samples from $F_\theta$ as in the inverse cdf approach, or for achieving "perfect" simulation. And for conducting inference by inverting the equation in $\theta$ into a distribution as in fiducial inference. Called by Efron Fisher's biggest blunder.

To relate with the previous answers, this [distributional] dependence of $X$ on [the true value of] $\theta$ does not imply that one can build a statistic that depends on $\theta$ because in the above equation both $\theta$ and $\xi$ are unobserved. Which is the whole point for conducting inference.