I am trying to learn the basics of Bayesian decision and I came across the phrase "proper prior" but I don't really understand what it means. Does anyone know?
Asked
Active
Viewed 5,801 times
8
-
1By default "prior" commonly means "proper prior", so this is a case where looking for the [antonym](https://en.wikipedia.org/wiki/Prior_probability#Improper_priors) may be more useful. (If this does not help, you can clarify, but consider adding the [tag:self-study] tag.) – GeoMatt22 Oct 08 '16 at 06:51
-
1I searched for the meaning of improper prior and I found that it means the prior does not integrate to one and can even be infinite (it is not a proper probability distribution so to speak...) Do you agree@GeoMatt22? Also, what about the self-sudy tag? – Learn_and_Share Oct 08 '16 at 07:08
-
1You are right, an improper prior is a prior that does not integrate to one and may even be infinite like e.g. a ''uniform prior'' over $[0,+\infty[$. – Oct 08 '16 at 07:29
-
@MedNait that is correct. A proper prior is literally a prior that is a PDF, so has unit integral. I mentioned self-study as your short question came across as similar to thse types of (homework/textbook) questions we see a lot, with little sense that the poster tried to solve on their own. But I realize that "proper prior" may be hard to Google, as noted in my original comment. – GeoMatt22 Oct 08 '16 at 08:20
1 Answers
7
A prior distribution that integrates to 1 is a proper prior, by contrast with an improper prior which doesn't.
For example, consider estimation of the mean, $\mu$ in a normal distribution. the following two prior distributions:
$\qquad f(\mu) = N(\mu_0,\tau^2)\,,\: -\infty<\mu<\infty$
$\qquad f(\mu) \propto c\,,\qquad\qquad -\infty<\mu<\infty.$
The first is a proper density. The second is not - no choice of $c$ can yield a density that integrates to $1$. Nevertheless, both lead to proper posterior distributions.
See the following posts which throw additional light on the use of improper priors issue and some closely related issues:
Flat, conjugate, and hyper- priors. What are they?
What is an "uninformative prior"? Can we ever have one with truly no information?
-
Just to complete my understanding, is a prior that integrates to 1 and is not always positive proper? – Learn_and_Share Oct 08 '16 at 11:46
-
That won't be any use at all, since it won't leave you with a proper posterior. – Glen_b Oct 08 '16 at 11:49
-
So basically, we can get a proper posterior even with an improper prior $p(\theta)$ because of the multiplication by the likelihood ($p(x|\theta)$: distribution of data given $\theta$ that cannot be improper!) that guaranties this. On the other hand, if the prior is negative valued nothing can correct for this. Is this correct? – Learn_and_Share Oct 08 '16 at 12:04
-
The product of a prior density that goes below with a likelihood will produce a posterior that goes below zero (unless the likelihood happens to be 0 there). But the prior going negative would also have no interpretation. An improper prior can generally still be understood in some (more or less vague) sense. – Glen_b Oct 08 '16 at 12:30
-
By: "An improper prior can still be understood in some sense" you mean as a weighting of the likelihood function, where the possibly high values of the improper prior can be corrected by the normalization factor $p(x)$? – Learn_and_Share Oct 08 '16 at 12:42
-