21

Stein's Paradox shows that when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean squared error) than any method that handles the parameters separately.

This is a very counterintuitive result. Does the same result hold if instead of using the $l_2$ norm (the expected mean squared error), we use the $l_1$ norm (the expected mean absolute error)?

Silverfish
  • 20,678
  • 23
  • 92
  • 180
Craig Feinstein
  • 403
  • 3
  • 8
  • 12
    It was harder than I thought: for instance, Das Gupta and Sinha (1997) establish a Stein effect under absolute error loss. – Xi'an Mar 24 '15 at 21:14
  • 3
    @Xi'an: This paper, right? http://www.stat.purdue.edu/research/technical_reports/pdfs/1997/tr97-22.pdf On p. 3 it says there's a Stein estimator that is "natural" for any $\alpha$-norm with $\alpha \geq 1$. And its form does not depend on $\alpha$. That is surprising to me - I always thought the Stein phenomenon was somewhat tied into the geometry of the $\ell_2$ norm. – Paul Sep 12 '15 at 13:49
  • 2
    @Paul: yes this is the paper. I think there is evidence in the literature that the Stein effect has little to do with the $\mathscr{l}_2$ norm, as it occurs in all kinds of settings, incl. non-Euclidean ones. – Xi'an Sep 12 '15 at 15:09

1 Answers1

3

Stein's paradox holds for all loss functions, and even worse- admissibility w.r.t. to a particular loss function probably implies inadmissibility w.r.t to any other loss.

For a formal treatment see Section 8.8 (Shrinkage Estimators) in [1].

[1] van der Vaart, A. W. Asymptotic Statistics. Cambridge, UK ; New York, NY, USA: Cambridge University Press, 1998.

JohnRos
  • 5,336
  • 26
  • 56
  • The inadmissability part seems to make sense. I always thought the Stein estimator was gaming the loss function to some extent. You pick a loss function, I pick some shrinkage that draws it down a little bit. – Paul Sep 19 '15 at 13:22