3

If the research question necessitates a more subjective-Bayesian approach to probability, i.e., betting odds, can Jaynes's system still work?

I ask this because I've read that one of the axioms for Jaynes's system is that all observers with the same information should choose the same prior.

pglpm
  • 1,175
  • 7
  • 18
jocol
  • 33
  • 3

2 Answers2

8

First, there is no such thing as a unique non-informative or objective or reference prior. You can check on different entries from this forum, here and there for instance. Hence, Jaynes' approach is a formal construct that applies in full generality [in the sense that it gives a methodology to derive a prior from a given collection of moment equations and the choice of a dominating measure] but this is not the only one and there is no absolute argument to prefer it to others.

Second, prior information is almost always a vague notion, at least from a probabilistic perspective and therefore there are many ways to turn it into a probability distribution. Once again, Jaynes' principle of maximum entropy allows for a somewhat generic translation of moment based information bits into a prior distribution. Other approaches apply as well, though.

Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • 1
    What do you mean by "full generality?" I eventually plan to the more falsificationist gelman style bayes for practice (has a frequency probability model that can be falsified by severe tests), but the only beginner level text i have approaches things from a Jaynes pov. Will this Jaynes foundation serve me well, or will i just get confused? – jocol Nov 20 '16 at 21:25
  • Here is the technical justification for the book: http://nbviewer.jupyter.org/github/bblais/Tech-SIE/blob/master/Prologue/Prologue.ipynb. – jocol Nov 20 '16 at 21:29
  • 1
    It's an applied stats book based om Jaynes called "Statistical inference for everyone" – jocol Nov 20 '16 at 21:33
  • 1
    Here is a paper comparing gelman to jaynes and others https://arxiv.org/abs/1508.05453. Is this an abstraction violation of Jaynes? – jocol Nov 20 '16 at 21:35
  • 1
    Thanks for the answer. So once the prior is derived through other means or from a different philsophy, can the rest of the Jaynes process in deriving the posterior probability still hold? I guess what im trying to get at is: is the jaynes process and knowledge generalizable to other bayesian methodologies? – jocol Nov 20 '16 at 23:07
  • I do not understand what you mean by "jaynes process and knowledge generalizable to other bayesian methodologies". Jaynes' maxent principle gives a way to derive priors informed by moment constraints. Reference priors à la Bernardo and Berger are driven by another principle that ranks the importance of components of the parameter, &tc, &tc – Xi'an Nov 21 '16 at 07:08
3

Yes, Jaynes's "system" is just the standard formulation of the probability calculus, it can be used in full generality. What changes is the interpretation of what's inside the conditional of a probability, i.e. what $B$ in $\mathrm{P}(A \mathpunct{\mid} B)$ represents. This in turn can affect whether a person accepts the relevance – as opposed to the formal correctness – of a probability calculation in a given problem.

The classics, e.g. besides Jaynes's book:

  • Johnson: Probability: The relations of proposal to supposal. Axioms. The Deductive and inductive problems (Mind 41:1–16, 281–296, 409–423; 1932)
  • Keynes: A Treatise on Probability (Macmillan 1957)
  • Ramsey: Truth and probability (in The Foundations of Mathematics, Routledge & Kegan Paul 1950, pp. 23–52)
  • Jeffreys: Scientific Inference (CUP 3rd ed. 1973)
  • Jeffreys: Theory of Probability (OUP 3rd ed. 2003)
  • Pólya: Heuristic reasoning and the theory of probability (Am. Math. Monthly 48:450, 1941)
  • Pólya: Preliminary remarks on a logic of plausible inference (Dialectica 3:28, 1949)
  • Koopman: The bases of probability (Bull. Am. Math. Soc. 46:763, 1940)
  • Koopman: The axioms and algebra of intuitive probability (Ann. Math. 41:269, 1940)
  • Good: Probability and the Weighing of Evidence (Griffin 1950)
  • de Finetti: Theory of Probability (Wiley 1990)
  • Savage: The Foundations of Statistics (Dover 2nd ed. 1972)
  • I also recommend Good: The Bayesian influence, or how to sweep subjectivism under the carpet (in Harper & Hooker: Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science II, Reidel 1976, pp. 125–174)

propose and motivate different interpretations, roughly ranging between two extreme positions:

  1. The numerical value of $\mathrm{P}(A \mathpunct{\mid} B)$ is unequivocally determined, at least in principle, by the propositions $A$ and $B$.
  2. The numerical value of $\mathrm{P}(A)$ is chosen even whimsically by the agent (provided it is coherent with other probability assignments by the same agent). In this case we could interpret the $B$ in $\mathrm{P}(A \mathpunct{\mid} B)$ as referring to the beliefs of that particular agent – which can be expressed by propositions. Conditionals as in $\mathrm{P}(A \mathpunct{\mid} C \land B)$ can be interpreted as stating that the particular agent added $C$ to their beliefs $B$.

I personally find the first position fascinating, but see no way to determine this unique numerical value, also because the content of $B$ is usually very vague. Regarding the second position, I think most people would object to a whimsical numerical assignment of a probability. We usually like to see a shred of reasoning and rationality behind a probability assignment. Either way, the basic rules for conjunction, disjunction, negation apply seamlessly in both cases.

A "neutral" position also seems possible: if we have a set of coherent assignments $\mathrm{P}(A_1 \mathpunct{\mid} B) = a_1$, $\mathrm{P}(A_2 \mathpunct{\mid} B) = a_2$, etc., we could implicitly define $B$ as that proposition or agent beliefs such that the above assignments hold. Now, another person may or may not agree with these assignments for the particular problem they apply to. This agreement or disagreement can be interpreted as that person's agreeing that $B$ is or isn't the correct proposition for that particular problem, or that person's sharing or not sharing the beliefs $B$. But the result is the same; in the end what counts is whether someone wants to use those probability assignments or not.

In propositional truth logic there's, in my view, a very similar situation or dilemma. In order to derive the truth or falsity of a proposition, we first need to give some axioms, that is, propositions assumed to be true (or false) without proof. This is beautifully visualized in Gentzen calculi, for example. How to choose these axioms is outside the domain of propositional logic itself. And different people may indeed argue for different axioms to be relevant in a specific problem. It seems to me that the conditional $B$ in $\mathrm{P}(A \mathpunct{\mid} B)$ has the same purpose: a sort of symbolic placeholder for the probability assignments assumed without proof (as Good said, "the axioms of probability can provide only relationships between probabilities and cannot manufacture a probability out of nothing"). Maybe there's more place for disagreement in the case of probability "axiomatic assignments" because we have a continuum choice instead of a simple binary one.

For this view I recommend

  • Hailperin: Sentential Probability Logic (Associated UP 1996)
  • Hailperin: Logic with a Probability Semantics (Lehigh UP 2011)

which presents the probability calculus in a way very similar to the truth calculus of propositions.

pglpm
  • 1,175
  • 7
  • 18