Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [steins-phenomenon]

Stein's phenomenon (paradox) states that when three or more parameters are estimated at the same time, there are more accurate estimators than the average over all observations.

Stein's phenomenon, or Stein's paradox, refers to a surprising finding made by Charles Stein in 1955: when three or more parameters are estimated simultaneously, then there are more accurate estimators than simply taking the average of all observations. One particular example of a more accurate estimator is given by the so called James-Stein estimator (1961), that handles all parameters together, instead of treating them separately.

James-Stein estimator is an example of shrinkage estimator.

27 questions
74
votes
5 answers

Unified view on shrinkage: what is the relation (if any) between Stein's paradox, ridge regression, and random effects in mixed models?

Consider the following three phenomena. Stein's paradox: given some data from multivariate normal distribution in $\mathbb R^n, \: n\ge 3$, sample mean is not a very good estimator of the true mean. One can obtain an estimation with lower mean…
amoeba
  • 93,463
  • 28
  • 275
  • 317
53
votes
2 answers

Intuition behind why Stein's paradox only applies in dimensions $\ge 3$

Stein's Example shows that the maximum likelihood estimate of $n$ normally distributed variables with means $\mu_1,\ldots,\mu_n$ and variances $1$ is inadmissible (under a square loss function) iff $n\ge 3$. For a neat proof, see the first chapter…
Har
  • 1,494
  • 11
  • 15
23
votes
2 answers

James-Stein estimator: How did Efron and Morris calculate $\sigma^2$ in shrinkage factor for their baseball example?

I have a question on calculating James-Stein Shrinkage factor in the 1977 Scientific American paper by Bradley Efron and Carl Morris, "Stein's Paradox in Statistics". I gathered the data for the baseball players and it is given below: Name, avg45,…
Anand
  • 1,092
  • 2
  • 11
  • 21
21
votes
1 answer

Why is the James-Stein estimator called a "shrinkage" estimator?

I have been reading about the James-Stein estimator. It is defined, in this notes, as $$ \hat{\theta}=\left(1 - \frac{p-2}{\|X\|^2}\right)X$$ I have read the proof but I don't understand the following statement: Geometrically, the James–Stein…
3x89g2
  • 1,366
  • 1
  • 11
  • 26
21
votes
1 answer

Does Stein's Paradox still hold when using the $l_1$ norm instead of the $l_2$ norm?

Stein's Paradox shows that when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean squared error) than any method that handles the parameters…
Craig Feinstein
  • 403
  • 3
  • 8
15
votes
5 answers

James-Stein shrinkage 'in the wild'?

I am taken by the idea of James-Stein shrinkage (i.e. that a nonlinear function of a single observation of a vector of possibly independent normals can be a better estimator of the means of the random variables, where 'better' is measured by squared…
shabbychef
  • 10,388
  • 7
  • 50
  • 93
14
votes
1 answer

Is there a connection between empirical Bayes and random effects?

I recently happened to read about empirical Bayes (Casella, 1985, An introduction to empirical Bayes data analysis) and it looked a lot like random effects model; in that both have estimates shrunken to global mean. But I have not read it…
anonymous
  • 141
  • 1
  • 3
11
votes
1 answer

James-Stein Estimator with unequal variances

Every statement I find of the James-Stein estimator assumes that the random variables being estimated have the same (and unit) variance. But all of these examples also mention that the JS estimator can be used to estimate quantities with nothing to…
exp1orer
  • 233
  • 2
  • 7
7
votes
1 answer

James-Stein Estimator with unequal variances (Ch. 2)

After studying James-Stein estimators for a few weeks and looking at many different sources I am stuck at trying to understand how Efron and Morris calculated the Toxoplasmosis example in their 1975 paper (p.314). I looked at the 1973 paper (p.43…
sdittmar
  • 206
  • 1
  • 9
7
votes
1 answer

Shrinkage of the eigenvalues

Assume we have $n$ samples $X_1,..., X_n$ which are independent and identically distributed with mean = 0 and unknown non-singular covariance matrix $M$. Each sample $X_i$ is a vector of size $p\times 1$. I want to apply the "Stein-Haff estimator"…
Christina
  • 755
  • 6
  • 17
6
votes
1 answer

When estimating population mean, how can one half of the sample mean have lower risk than the sample mean itself?

I read Efron and Morris (1977) Stein's Paradox in Statistics with interest yesterday and stumbled upon the statement that, if and only if the population mean is close to zero, than the risk (mean squared error, MSE) of using half of the sample mean…
user1885116
  • 2,128
  • 3
  • 23
  • 26
5
votes
1 answer

Stein's estimator normality assumption

The Stein's estimator assumes that data points are draws from a normal distribution, i.e., $Z_i \sim N(\mu_i, \sigma^2_i)$. By looking at different sources (Wikipedia, Efron,James-Stein Estimator with unequal variances) it seems that each $Z_i$ can…
JC1
  • 217
  • 1
  • 8
5
votes
1 answer

Stein's estimator vs James-Stein estimator

I read a lot of sources concerning stein's estimator and James-Stein estimator. Unfortunately, a lot of sources do not write the correct formulas of each estimator. And so I am now confused!! Kindly, can someone explain me in details the difference…
Christina
  • 755
  • 6
  • 17
5
votes
1 answer

Shrinkage estimation of Efron and Morris (1972)

I read this article: Article1 and which was refined by the second article Article2 that was considered as a generalization of the James-Stein estimator. In article 1 for example, they considered the following estimator: $\sum' = [a S^{-1} +…
Christina
  • 755
  • 6
  • 17
4
votes
1 answer

Is the effectiveness of seemingly unrelated regression an example of Stein's paradox?

The existence of Stein's Example prima facie appears similar to seemingly unrelated regression (SUR) insofar as simultaneously estimating multiple parameters seems more effective than training them separately. In the Wikipedia articles linked above…
DifferentialPleiometry
  • 2,274
  • 1
  • 11
  • 27
1
2