7

There is a famous principle in Bayesian that says: 'Yesterday’s posterior is today’s prior'. Lindley (2000).

Now, suppose there are three studies conducted chronologically. Based on Lindley's (2000) principle, is it possible to use a wide prior for the oldest study, and then use its posterior as prior for the study after it and so on to arrive at one final posterior for the most recent study?

Is this a good prior selection strategy for the most recent study?

P.S. In my mind as a non-stat, non-math person, somehow the above idea mixes up with meta-analysis, so are the above strategy and meta-analysis different?

enter image description here

rnorouzian
  • 3,056
  • 2
  • 16
  • 40
  • 4
    If the timing of the previous studies is not special (whatever that means) I'd be tempted to fit a Bayesian hierarchical model on the previous studies and use the posterior form that as prior for the new study, with possible discounting. – Frank Harrell Dec 22 '17 at 17:00
  • In the frequentist framework people use what is called cumulative meta-analysis. – mdewey Dec 22 '17 at 18:07
  • @mdewey, right, as I mention in **P.S** part of my question, I want to know is what I have in mind legitimate or there are other considerations that I should keep in mind? – rnorouzian Dec 22 '17 at 18:14
  • 1
    One consideration is that you have to make very sure the parameters you are estimating are really the same across studies. Even apparently minor changes in study design can result in the parameters themselves actually being different, in which case you have to think carefully about how you are going to use the posterior from the previous study. – jbowman Dec 22 '17 at 18:18
  • @jbowman, thank you so much, I owe this to you! Here, data is binomially distributed, but each experiment has a different `n` (# of trials) and and different `s` (# of successes). Can we still follow the *'Yesterday’s posterior is today’s prior'* principle and inform a new study using the posterior of a previous study as its prior? – rnorouzian Dec 22 '17 at 18:20
  • Yes, in that case, you definitely can, assuming of course that its the same process that's generating the successes in each experiment. – jbowman Dec 22 '17 at 18:26
  • @jbowman, as you know I'm non-stat, you mean the different studies share the same $p$ (as a parameter) and also $n$ (i.e., the data-generating process is the same)? But if that would not be the case what we can do? – rnorouzian Dec 22 '17 at 18:29
  • No, $n$ can vary. I mean, you can't go from a coin flipping experiment with true $p = 1/2$ to a dice-rolling experiment where $p(1 \text{ or } 2) = 1/3$, for example, even though both are binomial. – jbowman Dec 22 '17 at 18:32
  • @jbowman, oh I see your point. Your concern is with sameness (for the lack of a better word) of the parameter across the studies? – rnorouzian Dec 22 '17 at 18:35
  • Yes, exactly so. – jbowman Dec 22 '17 at 18:38
  • @jbowman, if I may, are the above strategy (i.e., *Lindley's principle*) and meta-analysis different from each other? – rnorouzian Dec 22 '17 at 19:17
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/70696/discussion-between-jbowman-and-rnorouzian). – jbowman Dec 22 '17 at 19:20
  • @FrankHarrell, thanks Frank, given that the data in the above case is binomially distributed (I have # of trials from each study as well as # of successes), could you possibly suggest an `R` package to allow me to *"fit a Bayesian hierarchical model on the previous studies and use the posterior form that as prior for the new study, with possible discounting"*? – rnorouzian Dec 25 '17 at 21:06
  • I recommend the R `rstan` package that uses the `Stan` system. But first look at the R `brms` package that provides a nice wrapper for many models to allow R notation to be used to run `Stan`. There may be other R packages that are specifically for hierarchical models with `Stan`. This approach allows a lot of generality, e.g. odd-shaped posteriors, discounting the posterior by mixing it with a skeptical prior for the new study. – Frank Harrell Dec 26 '17 at 13:12
  • @FrankHarrell, for some reason `brms` doesn't get installed on my pc. But so, what is the source of uncertainty that can be modeled here in `rstan`? – rnorouzian Dec 27 '17 at 00:09
  • The Bayesian posterior will take into account all sources of uncertainty that you admit to in the model, including uncertainty about the variance of random effects. – Frank Harrell Dec 27 '17 at 13:50

2 Answers2

2

"Yesterday’s posterior is today’s prior" is the best Bayesian learning strategy if you know with absolute certainty that "Today's parameter is yesterday's parameter".

is it possible to use a wide prior for the oldest study, and then use its posterior as prior for the study after it and so on to arrive at one final posterior for the most recent study?

Yes as long as you know you are making inferences about the same (unknown) parameter:

  • same model
  • same experimental conditions (including sampling from the same population)
  • no deviation due to time depending or local phenomena

Note that if the lines in each dataset are considered to be independent of each other, the final posterior you get is the same as when considering all studies as a whole: merging all datasets into one (just basic copy/paste).

An interesting case in which such a simplifying assumption may hold or not is the Kalman Filter (or more generally Bayes filters): you acquire information at each observation making the prior dynamically evolve as $prior_{t+1}=posterior_t$.

But if at the same time, some random process is disturbing the parameter (known as "state" in Kalman filters), then the prior must be updated too due to this process. Your prior narrows down at each observation, but between two observations it broadens due to random changes.

In that case the prior you would use in the next study would be a broadened version of the posterior of the previous study. How much depends on the random dynamics and is very complicated, thus rarely done in practice.

Benoit Sanchez
  • 7,377
  • 21
  • 43
  • Lindsey's principle holds if "today's parameter is yesterday's parameter". It is the simplest form of meta-analysis. Meta-analysis is more general and potentially allows to take parameter variation into account. About Frank's comment, I don't enough about hierarchical models. – Benoit Sanchez Dec 23 '17 at 17:58
1

There has been some work on deriving priors from previous studies. Two relevant papers may be "Summarizing historical information on controls in clinical trials" by Neuenschwander and colleagues available from Clinical Trials and "Robust meta-analytic-predictive priors in clinical trials with historical control information" by Schmidli and colleagues available from Biometrics. It is difficult to summarise them so I give below the abstract of the Schmidli one.

Summary. Historical information is always relevant for clinical trial design. Additionally, if incorporated in the analysis of a new trial, historical data allow to reduce the number of subjects. This decreases costs and trial duration, facilitates recruitment, and may be more ethical. Yet, under prior-data conflict, a too optimistic use of historical data may be inappropriate. We address this challenge by deriving a Bayesian meta-analytic-predictive prior from historical data, which is then combined with the new data. This prospective approach is equivalent to a meta-analytic-combined analysis of historical and new data if parameters are exchangeable across trials. The prospective Bayesian version requires a good approximation of the metaanalytic- predictive prior, which is not available analytically. We propose two- or three-component mixtures of standard priors, which allow for good approximations and, for the one-parameter exponential family, straightforward posterior calculations. Moreover, since one of the mixture components is usually vague, mixture priors will often be heavy-tailed and therefore robust. Further robustness and a more rapid reaction to prior-data conflicts can be achieved by adding an extra weakly-informative mixture component. Use of historical prior information is particularly attractive for adaptive trials, as the randomization ratio can then be changed in case of prior-data conflict. Both frequentist operating characteristics and posterior summaries for various data scenarios show that these designs have desirable properties. We illustrate the methodology for a phase II proof-of-concept trial with historical controls from four studies. Robust meta-analytic-predictive priors alleviate prior-data conflicts - they should encourage better and more frequent use of historical data in clinical trials.

There is currently an R package RBesT available to do the meta-analytic prior. It is on CRAN

mdewey
  • 16,541
  • 22
  • 30
  • 57