5

The Stein's estimator assumes that data points are draws from a normal distribution, i.e., $Z_i \sim N(\mu_i, \sigma^2_i)$.

By looking at different sources (Wikipedia, Efron,James-Stein Estimator with unequal variances) it seems that each $Z_i$ can be drawn from a normal distribution with arbitrary mean and variance.

In the extreme, all means and variances are different: $$\mu_i\neq\mu_j \text{ and } \sigma^2_i\neq\sigma^2_j, \forall\ i\neq j$$

"A quirky example would be estimating the speed of light, tea consumption in Taiwan, and hog weight in Montana, all together."

Assuming my assumptions so far are correct, how does pushing each $Z_i$ towards the global mean can create a better individual predictor. If not, what is wrong with my assumptions?

Community
  • 1
JC1
  • 217
  • 1
  • 8

1 Answers1

2

The Stein estimators, like the James-Stein version $$\delta_a(\mathbf{z})=\mathbf{\mu}_0+\left(1-\frac{a}{[\mathbf{z}-\mathbf{\mu}_0]^T\Sigma^{-1}[\mathbf{z}-\mathbf{\mu}_0]}\right)[\mathbf{z}-\mathbf{\mu}_0]$$do not produce individually better estimators but globally better estimators, in the sense that the error $$L(\delta,\mathbf{\mu})=[\delta-\mathbf{\mu}]^T\Sigma^{-1}[\delta-\mathbf{\mu}]$$is smaller in average when using $\delta_{p-2}$ than when using $\delta_0$, whatever $\mathbf{\mu}_0$ is. There are components of $\mathbf{\mu}$ that fare worse under the new estimator, e.g., the speed of light estimate, and others that fare better, e.g., the weight of hogs.

Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • Thanks. I understand that the Stein's estimator minimizes the MSE. But in this case it seems very unintuitive that it should be a better predictor than $Z$ itself. Aren't these quantities ($Z_i$) completely unrelated? – JC1 Mar 04 '16 at 05:46
  • 1
    This is why it is called the Stein paradox. But in practice, the gain in MSE can be negligible due to either the choice of a poor central location $\mu_0$ or to inappropriate scales. Intuitively the gain mostly occurs near the location $\mu_0$. – Xi'an Mar 04 '16 at 10:04
  • Let's say in a hypothetical situation you needed to estimator the acreage of the homestead as well as the length and width. For this problem we're assuming that all land owned is rectangles. Let's say the customer is most interested in the length and width. Could the estimation for the acreage be used to help correct or reduce the error in the length and width models? – Chris Mar 15 '16 at 14:43
  • 1
    @Chris: I do not understand your question, maybe due to my poor English: if you add the acreage to length and width, the third "variable" is directly and strongly connected with the other two, so you are not adding a third dimension. – Xi'an Mar 15 '16 at 20:35