0

I am a beginner to Bayesian analysis and I am trying to fit a logistic regression model using Bayesian approach. For the prior distribution of the $\beta$ regression coefficients , I used a non informative prior which is normally distributed with a very high variance. $\beta \sim N(0,10000)$ . For the analysis ,MCMC method was employed in JAGS package in R.

I have seen in several books and articles that , when we use non informative priors, the results would be similar to classical method.

But in my case when I compare the results of my Bayesian logistic regression model with the classical logistic regression model , it is not similar.

What may be the reason for this ?

My main intention of this question is to learn the theoretical reasoning behind my problem and what kind of steps that I should follow when there is a problem this.

Thank you.

student_R123
  • 751
  • 4
  • 13
  • There may be several reasons why this JAGS trick does not work: (a) 10⁵ is only "large" when $\beta$ is small compared with 10⁵, the flat prior should be used instead (b) the MCMC sampler may fail to converge. You should check the code with a simulated dataset where you know $\beta$. Note that we process the logistic regression model in [our book](https://link.springer.com/chapter/10.1007/978-1-4614-8687-9_4) if you want a textbook illustration of the Bayesian non-informative analysis. – Xi'an Aug 27 '20 at 05:20

1 Answers1

2

There are several reasons for using non-informative priors. These include:

(a) Actually not having any useful prior information or strong personal opinion upon which to base an informative prior.

(b) Trying to show skeptics that a Bayesian approach is reasonable because a non-informative prior gives a result numerically similar to a traditional (frequentist) approach when little or no prior information is provided, while allowing for use of prior information when it exists.

(c) A sensitivity analysis (often ad hoc) to see how much influence a strong prior has had on the results of a Bayesian analysis.

(d) Because a Bayesian approach is simpler than a traditional one, so that one wants to use the Bayesian approach even when no useful prior information is available.

(e) In a continuing investigation, to get onto a Bayesian track initially without prior information, but hoping to use results of phase $i-1$ of the investigation as prior information for phase $i,$ and so on as the investigation moves iteratively from one phase to the next.

As an example of (d), it is now widely recognized that Wald confidence intervals of the type $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}$ have such bad coverage probabilities as to be essentially useless for $n$ smaller than $n=100$ or so. There are several other styles of CIs that are better, some of which are quite messy computationally. However, the Jeffries CI [based on a Bayesian procedure with the non-informative prior distribution $\mathsf{Beta}(.5,.5)]$ works quite well in frequentist situations and is easy to compute in R. Specifically, if there are 43 Successes in 63 binomial trials, then a 95% interval estimate for $p$ is $(0.561,\,0.787)$. [Compared with Wald $(0.567,\, 0.797)$ and Agresti-Coull $(0.561,\, 0.787)].$

qbeta(c(.025,.975), 43.5, 20.5)
[1] 0.5614128 0.7873252

Perhaps an example of (b) is in this Q&A, where a traditional method fails only in occasional difficult cases, whereas a Bayesian method with a Gibbs Sampler provides a common method for all cases.

However, you have to be careful about "noninformative" priors because as the situation changes what you imagined to be noninformative, can be more informative than you supposed in a not-so-similar situation. (See the note by @Xi'an.)

BruceET
  • 47,896
  • 2
  • 28
  • 76