2

I am trying to reproduce a Gibbs sampling but some points are unclear to me. Let's assume a linear model of the following form $$y=x\beta + \epsilon$$

The prior distribution for the beta sensitivities is normal, while the prior distribution for the error term epsilon is gamma.

Can you provide me the intuition of this the prior gamma distribution (and not normal)?

Alexis
  • 26,219
  • 5
  • 78
  • 131
branchwarren
  • 247
  • 2
  • 8
  • Are you sure it's not $y \sim \mathcal{N}(x\beta, \tau)$ where for $\tau$ gamma prior is used..? – Tim Feb 02 '17 at 12:11
  • @Tim you are right, sorry i am completely new on this field and my background is limited. can you provide me further reference or explanation – branchwarren Feb 02 '17 at 12:17
  • I assume you would not use a gamma distribution here, because this would constrain $\epsilon$ to be $>0$, while $x\beta$ could be negative. – Björn Feb 02 '17 at 13:11

1 Answers1

7

What you are describing is a simple regression model

$$ Y = X\beta + \varepsilon $$

that can be alternatively described as

$$ \begin{align} \mu &= X\beta \\ Y &\sim \mathcal{N}(\mu, \tau) \end{align} $$

saying it differently, you estimate conditional mean of $Y$

$$ E(Y|X) = \mu $$

and assume normally distributed errors

$$ \varepsilon \sim \mathcal{N}(0, \tau) $$

what leads to $Y = \mu + \varepsilon$.

As about priors, normal distribution is a perfectly fine prior for $\beta$, but for precision parameter you need prior with positive support, since precision (as alternative to variance) need to be positive (there is nor such a thing as negative variance). Gamma fits this requirements and, moreover, is a conjugate prior for precision parameter of normal distribution, what makes computation easier since closed-form solution is directly available (see here for example using variance rather then precision).

This has nothing to do with Gibbs sampling, but it is about formulation of the model.

Community
  • 1
Tim
  • 108,699
  • 20
  • 212
  • 390
  • thanks for these references and for your expalanation, very helpful – branchwarren Feb 02 '17 at 12:57
  • Am I correct in thinking that the values that are sampled using the full conditional gamma distribution will be values of the precision? Will the empirical mean of these samples then estimate precision of the error term $\epsilon$ rather than the variance? – samvoit4 Jun 18 '19 at 22:52
  • 1
    @samvoit4 yes, gamma is a conjugate prior for precision, inverse gamma is a prior for variance. – Tim Jun 19 '19 at 07:54