0

The further I read Bayesian books, the clearer it becomes that traditional Bayesian inference has focused on very narrow problems: it requires users to completely specify a prior distribution over parameters.

Most of the time, however, users have only a coarse knowledge of reality. For instance, $\theta \in [a, b]$.

If my prior information is simply that $\theta \in [a, b]$, and I have observed some data $X\sim P(X|\theta)$, what is the rational way to update my beliefs about $\theta$?

  • Is that not just a uniform distribution on the interval? – Dave Mar 19 '21 at 17:39
  • 1
    @Dave no, I don't have any knowledge of the relative probability of $\theta$ inside the interval. And as shown here (https://stats.stackexchange.com/questions/514557/what-is-the-wine-water-paradox-in-bayesian-statistics-and-what-is-its-resolutio) lack of knowledge is not the same as a uniform distribution. –  Mar 19 '21 at 17:40
  • 1
    Are you sure you don't have any beliefs about $\theta$ beyond "$\theta \in [a, b]$"? Imagine you were forced to bet one dollar either for or against the proposition that $\theta \in [a + \frac{13(b-a)}{100}, a + \frac{14(b-a)}{100}]$. If you picked "against", that would suggest that you assign prior probability < 0.5 to the propositon. If you picked "for", then you probably lost a dollar, because most of the time, when you know a value is in a particular interval, it isn't between 13% and 14% along the interval. – fblundun Mar 19 '21 at 21:11
  • @fblundun ok, say you give me this bet. Then I say I'm indifferent since you are forcing me. Then you give me another bet which "logically" I should take, given that I was indifferent to to that previous bet. But I say again I'm indifferent since you are forcing me. There will probably be no probability function that explain my bets. –  Mar 19 '21 at 21:51
  • @bayesian_newbie by "indifferent" do you mean you pick between "for" and "against" at random? In that case my point is that being indifferent to that proposition isn't a valid position, it's just a mistake on your end which will probably cost you if you randomly bet "for" rather than making the correct choice of betting "against". – fblundun Mar 19 '21 at 22:41

1 Answers1

0

Not quite the same but Bayes Linear (BL) Statistics tells us how to do bayesian analysis if we only specify means and variances (and covariances). I.e. we are only making limited statements about our beliefs. It is also advantageous over a "full" bayesian analysis in the sense that it is entirely tractable and therefore much faster than computational bayesian methods.

If you specify $\theta \in [a,b]$ then under a BL framework you might say $E(\theta) = m = 0.5(a+b)$ (the mid point). You would also then need to specify the variance. You might say that $m + k \times var(\theta) = b$. A sensible choice would be $k=3$ by Pukelshiems $3\sigma$ rule.

You can then perform BL updates in light of data which give you an updated mean and variance of $\theta$. This isn't the same as a posterior distribution but is a pair of posterior moments.

Essentially, you don't get as much information as a full bayes analysis but that's because you're not putting as much in

jcken
  • 2,092
  • 5
  • 17
  • I'm not sure I understand this, but knowing this exists is already a plus, so +1 for you even though this doesn't directly answer the question. –  Mar 19 '21 at 21:46
  • I think its the closest we can get to your question, I.e. an incomplete prior specification. In practice people often choose priors which roughly match their beliefs because with data the posterior will be roughly the same for many choices of prior – jcken Mar 19 '21 at 22:03