3

I have just worked through an example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator, leading me to believe that improper priors are uninformative. But must this be true?

Xi'an
  • 90,397
  • 9
  • 157
  • 575
DavidSilverberg
  • 739
  • 6
  • 18
  • Can you define what improper prior is? It is a little ambiguous. – Bill Chen Dec 04 '19 at 17:38
  • 4
    I've always taken "improper" to mean any prior that isn't a valid probability distribution. Do you mean a *uniform* prior? – shadowtalker Dec 04 '19 at 18:10
  • 1
    It is not clear to me that the *Bayesian estimator* should be the mode of the posterior distribution as I cannot see a realistic loss function which justifies this (except perhaps for saying that the loss for any error does not depend on the size of the error) – Henry Dec 04 '19 at 18:46
  • I’m talking about a prior distribution that is not a true distribution. – DavidSilverberg Dec 04 '19 at 20:39
  • @Henry: [I agree](https://xianblog.wordpress.com/tag/map-estimators/) on the dubious status of MAP estimators as true Bayesian estimators. – Xi'an Dec 04 '19 at 21:04
  • 1
    My concern was not with the word "improper" which I think is clear enough, but rather with *informative*; what would convince you that a given prior was informative? – Glen_b Dec 05 '19 at 07:50

2 Answers2

10

...example where, with an improper prior, the Bayesian estimator equals the maximum likelihood estimator...

There is a fundamental issue with this property, namely that it depends on the parameterisation of the sampling model. Indeed, if $$\hat\theta^\text{MAP}=\arg\max_\theta L(\theta|x)\pi(\theta)=\arg\max_\theta L(\theta|x)=\hat\theta^\text{MLE}\tag{1}$$ consider the change of parameterisation $\eta=h(\theta)$, where $h$ is a bijection. Then the prior on $\eta$ is $$\tilde\pi(\eta)=\pi(h^{-1}(\eta))\ \underbrace{\left|\dfrac{\text{d}h^{-1}(\eta)}{\text{d}\eta}\right|}_\text{Jacobian}$$ and there is no reason for $$\hat\eta^\text{MAP}=\arg\max_\eta L(\eta|x)\tilde\pi(\eta)=\arg\max_\eta L(\eta|x)=\hat\eta^\text{MLE}$$ to hold, even though $$\hat\eta^\text{MLE}=h(\hat\theta^\text{MLE})$$ because of the Jacobian. MAP estimators are not invariant by reparameterisation.

As an additional remark, that a particular prior leads to this (dubious) identity is not one of the definitions (here, there, or there) of non-informative priors, since, for one thing, being non-informative is a property that depends on both the data (I assume (1) holds for a particular $x$) and the parameterisation. Furthermore, improper priors should not be assimilated to non-informative priors.

Xi'an
  • 90,397
  • 9
  • 157
  • 575
9

The prior which is uniform over $[m,\infty)$ is improper, but informative: it contains the information that the value is at least $m$.

user54038
  • 493
  • 3
  • 9