Questions tagged [sufficient-statistics]

A sufficient statistic is a lower dimensional function of the data which contains all relevant information about a certain parameter in itself.

Consider $n$ independent observations from a random variable $X: X_1, X_2,...,X_n$ such that $f(x_1,...,x_n|\theta)=\prod_{i=1}^{n}p(x_{i}|\theta)$ where $\theta$ is a parameter to be estimated. A sufficient statistic reduces the whole data into a function of $x$ that carries all relevant information about $\theta$.

As an example let $X \sim \operatorname{Exponential}(\lambda)$ and suppose $x_i > 0$, then the joint probability $$f(x_1,...,x_n|\lambda) = \prod_{i=1}^{n} \lambda e^{-\lambda x_{i}}$$ can be written as $$f(x_1,...,x_n|\lambda) = \lambda^{n} e^{-\lambda \sum x_{i}}$$ reducing the data to the sufficient statistic $\sum x_{i}$. Knowing the sample average and the number of observations then allows us to calculate an estimate of $\lambda$ without considering the whole data vector $x_i$.

407 questions
32
votes
3 answers

How can a statistician who has the data for a non-normal distribution guess better than one who only has the mean?

Let's say we have a game with two players. Both of them know that five samples are drawn from some distribution (not normal). None of them know the parameters of the distribution used to generate the data. The goal of the game is to estimate the…
ryu576
  • 2,220
  • 1
  • 16
  • 25
30
votes
2 answers

Why do we not care about completeness, sufficiency of an estimator as much anymore?

When we begin to learn Statistics, we learn about seemingly important class of estimators that satisfy the properties sufficiency and completeness. However, when I read recent articles in Statistics I can hardly find any papers that address complete…
29
votes
1 answer

When if ever is a median statistic a sufficient statistic?

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the sample mean, I cannot think of another…
Xi'an
  • 90,397
  • 9
  • 157
  • 575
28
votes
3 answers

Sufficient statistics for layman

Can someone please explain sufficient statistics in very basic terms? I come from an engineering background, and I have gone through a lot of stuff but failed to find an intuitive explanation.
17
votes
2 answers

Jointly Complete Sufficient Statistics: Uniform(a, b)

Let $\mathbf{X}= (x_1, x_2, \dots x_n)$ be a random sample from the uniform distribution on $(a,b)$, where $a < b$. Let $Y_1$ and $Y_n$ be the largest and smallest order statistics. Show that the statistic $(Y_1, Y_n)$ is a jointly complete…
17
votes
3 answers

Why a sufficient statistic contains all the information needed to compute any estimate of the parameter?

I've just started studying statistics and I can't get an intuitive understanding of sufficiency. To be more precise I can't understand how to show that the following two paragraphs are equivalent: Roughly, given a set X of independent identically…
16
votes
1 answer

Sufficient statistic, specifics/intuition problems

I'm teaching myself some statistics for fun and I have some confusion regarding sufficient statistics. I'll write out my confusions in list format: If a distribution has $n$ parameters then will it have $n$ sufficient statistics? Is there any sort…
Kimchi
  • 161
  • 4
13
votes
1 answer

Intuitive understanding of the Halmos-Savage theorem

The Halmos-Savage theorem says that for a dominated statistical model $(\Omega, \mathscr A, \mathscr P)$ a statistic $T: (\Omega, \mathscr A, \mathscr P)\to(\Omega', \mathscr A')$ is sufficient if (and only if) for all $\{P \in \mathscr{P} \} $…
12
votes
1 answer

Do the mean and the variance always exist for exponential family distributions?

Assume a scalar random variable $X$ belongs to a vector-parameter exponential family with p.d.f. $$ f_X(x|\boldsymbol \theta) = h(x) \exp\left(\sum_{i=1}^s \eta_i({\boldsymbol \theta}) T_i(x) - A({\boldsymbol \theta}) \right) $$ where ${\boldsymbol…
Wei
  • 345
  • 1
  • 8
11
votes
1 answer

What is "Likelihood Principle"?

While I was studying "Bayesian Inference", I happen to encounter the term, "Likelihood Principle" but I don't really get the meaning of it. I assume it is connected to "Bayesian Inference" but I am not really sure what it is and what part of…
11
votes
2 answers

How does Bayesian Sufficiency relate to Frequentist Sufficiency?

The simplest definition of a sufficient statistics in the frequentist perspective is given here in Wikipedia. However, I recently came across in a Bayesian book, with the definition $P(\theta|x,t)=P(\theta|t)$. It's stated in the link that both are…
11
votes
1 answer

Sufficiency or Insufficiency

Consider a random sample $\{X_1,X_2,X_3\}$ where $X_i$ are i.i.d. $Bernoulli(p)$ random variables where $p\in(0,1)$. Check if $T(X)=X_1+2X_2+X_3$ is a sufficient statistic for $p$. Firstly, how can we find the distribution for…
Landon Carter
  • 1,295
  • 11
  • 21
10
votes
2 answers

Solution to German Tank Problem

Is there a formal mathematical proof that the solution to the German Tank Problem is a function of only the parameters k (number of observed samples) and m (maximum value among observed samples)? In other words, can one prove that the solution is…
10
votes
1 answer

Are unbiased efficient estimators stochastically dominant over other (median) unbiased estimators?

General description Does an efficient estimator (which has sample variance equal to the Cramér–Rao bound) maximize the probability for being close to the true parameter $\theta$? Say we compare the difference or absolute difference between the…
10
votes
1 answer

Proof of Pitman–Koopman–Darmois theorem

Where can I find a proof of Pitman–Koopman–Darmois theorem? I have googled for quite some time. Strangely, many notes mention this theorem yet none of them present the proof.
1
2 3
27 28