Setting a proper prior distribution on sigma for Bayesian Regression

Question

I'd like to know how to set a proper prior distribution on sigma when using JAGS in a Bayesian regression.

Here is what I have:

model{
    for( i in 1 : N ) {
        y[i] ~ dnorm(y_hat[i], tau )
        y_hat[i] <- inprod(b[1:K], x[i,1:K])  
  }   

    sigma ~ dunif(0,10000)
    #tau ~ dgamma(.001,.001)
    tau <- pow(sigma,-2)

    for(j in 1:K) {
      b[j] ~ dnorm(1/K,4*K*K)    
    }
}

K is the number of predictor variables I have.

If I have a y[i] with known range of 0 to 1 (or any known range), then how could I use that knowledge to change the distribution I've chosen on sigma? It seems a bit crazy to choose a prior distribution like I've done when there's no chance in the world that it would be anywhere close to 10,000.

I don't know jags, but inverse gamma priors are commonly used for $\sigma^2$ in a normal model. — Glen_b, May 29 '14 at 05:24
Beta distribution is good for ranged data. Use it instead of normal. — probabilityislogic, May 29 '14 at 07:41
Since $\sigma^2$ represents the variance of the residuals, I think I can safely put a cap on the distribution of $\sigma^2$ by using the total variance on y. — Anthony Tyler, May 29 '14 at 14:49

score 1 · Answer 1 · edited Apr 13 '17 at 12:44

Since your data is bounded, your variance will be bounded above. See Maximum value of coefficient of variation for bounded data set. Therefore you should choose a prior for $\sigma^2$ that, at the very least, has support only from 0 to that upper bound.

Also, once you've chosen a prior, you can validate your choice by comparing the prior predictive distribution to the corresponding (marginal) distribution of your data.

score 0 · Answer 2 · answered May 29 '14 at 11:00

Just a few remarks.

In general, if $N$ is large enough then vague priors can't harm, because the posterior distribution is shrunk towards the likelihood.

Strictly speaking, looking at your data to choose your prior distribution is not "fully Bayesian", because your priors should come from previous knowledge. However, in the empirical Bayes approach (see Carlin and Louis, chap. 5) the observed data are used to estimate the prior distribution and then proceed as though the prior were known.

At last, you could look at Gelman, who recommends weakly-informative prior distributions, and a folded-noncentral-t or a half-Cauchy instead of the inverse-gamma prior for $\sigma^2$.

I noticed that the vague prior doesn't harm with large sample sizes, but my sample sizes are not always large. Playing around with changing my prior does seem to affect the results in these cases. The empirical Bayes approach is intriguing. — Anthony Tyler, May 29 '14 at 14:37

Setting a proper prior distribution on sigma for Bayesian Regression

2 Answers2