3

Is there an admissible estimator for a linear regression model with many parameters without restricting the parameter space?

Admissibility will be with respect to Mean Square Error on the regression parameter vector.

It seems that James-Stein estimators beat OLS, but are still inadmissible.

Thanks

Xi'an
  • 90,397
  • 9
  • 157
  • 575
Cagdas Ozgenc
  • 3,716
  • 2
  • 29
  • 55

2 Answers2

13

[I will use the $x\sim\mathcal{N}_p(\mu,I_p)$ case instead of the equivalent linear regression model in order to simplify notations.]

Since the family of James-Stein estimators$$\delta_a(x)=\left(1-\frac{a}{||x||}\right)^+x$$all have a strictly smaller mean square error (risk) than $\delta_0$ when $0<a<2(p-2)$, $\delta_0$ is not admissible. While the $\delta_a$ themselves are also inadmissible because they are not analytic functions of $x$, they are dominated by admissible estimators that are either proper Bayes or limits of proper Bayes estimators, by virtue of the complete class theorem (see, e.g., Section 8.3.3 of my book). Hence $\delta_0$ is inadmissible and dominated by admissible estimators. For instance, see Maruyama (2004) for a class of admissible minimax (hence dominating $\delta_0$) estimators. And Berger and Robert (1990), and Maruyama and Strawderman (2006) for classes of minimax generalised Bayes estimators.

Here is the main complete class result:

Theorem 8.3.9 (Stein's necessary and sufficient condition) Under the hypotheses

  1. $f(x|\theta)$ is continuous in $\theta$ and strictly positive on $\Theta$; and
  2. the loss $\rm{L}$ is strictly convex, continuous and, if $E\subset\Theta$ is compact,$$\lim_{\|\delta\|\rightarrow +\infty} > \inf_{\theta\in E} \rm{L}(\theta,\delta) =+\infty$$

an estimator $\delta$ is admissible if, and only if, there exist a sequence $(F_n)$ of increasing compact sets such that $\Theta=\bigcup_n F_n$, a sequence $(\pi_n)$ of finite measures with support $F_n$, and a sequence $(\delta_n)$ of Bayes estimators associated with $\pi_n$ such that

  1. there exists a compact set $E_0\subset \Theta$ such that $\inf_n \pi_n(E_0) \ge 1$
  2. if $E\subset \Theta$ is compact, $\sup_n \pi_n(E) > <+\infty$
  3. $\lim_n r(\pi_n,\delta)-r(\pi_n) = 0$
  4. $\lim_n R(\theta,\delta_n)= R(\theta,\delta)$.
Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • 2
    The theorem is too complex for me to understand. But I see that it is working on compact domain avoiding my question. – Cagdas Ozgenc Apr 05 '15 at 16:34
  • 4
    If you read carefully enough the assumptions and understand the maths, you should be able to see that Stein's condition above does not require $\Theta$ to be compact. – Xi'an Apr 05 '15 at 16:35
  • 5
    Dear Cagadas: your statement that "Bayesian is admissable (sic) only by wrong definition" is complete nonsense! – Zen Apr 20 '15 at 17:08
  • @Xi'an are there any known admissible estimators that dominate the (positive part or plain) James-Stein estimator? – user795305 Sep 13 '17 at 21:53
  • 1
    Check the papers by [Maruyama and Strawderman](https://rd.springer.com/article/10.1007/BF02506885). – Xi'an Sep 14 '17 at 06:37
0

Well if you only care about your estimator being admissible, any trivial estimator will do the job...

If you also want convergence, true admissiblity seems pretty complicated. I think asymptotical admissibility is the best you can get (look for results like Pinsker's theorem).

Adrien
  • 478
  • 4
  • 9
  • 1
    As indicated in my answer, there exist minimax admissible estimators in the case the dimension of the regression parameter is large than 2. If this is not good enough, please indicate which optimality criterion you are looking for, besides admissibility. – Xi'an Apr 06 '15 at 09:15
  • @Xi'an I wasn't looking for some specific optimality criterion. I was trying to learn about the approaches to this problem in general. Thank you for pointing out the Maruyama estimators. When estimators are not restricted to unbiased estimators, is minimax the general approach, or are there any other approaches? – Cagdas Ozgenc Apr 06 '15 at 12:00
  • 3
    The only decision-theoretic approaches I can provide are minimaxity (against a specific loss function), admissibility (against a specific loss function), and invariance (against a specific action group). – Xi'an Apr 06 '15 at 12:56