2

I have been reading about Bayesian inference by Han Liu and Larry Wasserman. In the section 12.2.3 they defined a bayesian inference on a variable parameterised by a function.

Given a random variable $X \sim Berouli(\theta)$ and $D_n = \{X_1,X_2,...X_n\}$ the set of observed data, and $\psi = log(\frac{\theta}{1 - \theta})$. Also let $\pi(\theta) = 1$, then posterior distribution for $\theta$ is equal to a $Beta \sim (S_n + 1, n-S_n +1)$ distributed, where $S_n = \sum_{i=1}^nX_i$, the number of successes.

The posterior is $$p(\theta|D) = \frac{\Gamma(n+2)}{\Gamma(S_n+1)\Gamma(n-S_n+1)}\theta^{S_n}\theta^{n - S_n}$$We can also find the posterior of $\psi$ by substituting $\theta$ with $\psi$ to get
$$p(\psi|D) = \frac{\Gamma(n+2)}{\Gamma(S_n+1)\Gamma(n-S_n+1)}({\frac{e^{\psi}}{1+e^{\psi}}})^{S_n}(\frac{e^{\psi}}{1+ e^{\psi}})^{n - S_n}$$

To sample from $p(\psi|D)$ we can sample from $p(\theta|D)$ and compute $\psi$ to obtain samples for $p(\psi|D)$.

Although this question may seem stupid.. I would like to know where such instances of computing posterior of functions of random variables being used in Bayesian Inference ?

Also, another point im not sure is why the authors decided to define an equation for the posterior CDF of the function $\tau = g(\theta)$. Why are we interested in a posterior CDF ?

calveeen
  • 746
  • 1
  • 10

1 Answers1

4

Bayesian inference can be used in any scenario where you could use other forms of statistical inference, e.g. maximum likelihood. Additionally, it has some extra advantages, since it allows you for using priors, so for bringing out-of-data information into the model, and it gives you uncertainty estimates for free, since you learn the distribution of the parameters and predictions as a result. In statistics, we often are interested in transformed parameters, e.g. your data records temperature in Fahrenheit degrees, and you want the result in Celsius degrees scale, so you transform them.

As about your second question, by applying Bayes theorem we obtain a probability density function, not cumulative distribution function. We want to learn it to know what values of parameters are more likely, than others. In some cases, you may want to calculate cumulative probabilities to answer questions like “what is the probability that the parameter is smaller than some value”.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • Bayesian inference for transformed variables is kind of trivial in the sense that u can just convert ur posterior samples for actual variables into the transformed variables and vice-versa ? – calveeen Oct 13 '20 at 06:33
  • The book in section 12.2.3 talks about the posterior cdf of the transformed variable though.. Quote "Given the data $D_n = {X_1, . . . , X_n}$, how do we make inferences about a function $\tau = g(\theta)$? The posterior CDF for $\tau$ is ..." – calveeen Oct 13 '20 at 06:35
  • @calveeen yes to first comment. As about second, I edited my answer. – Tim Oct 13 '20 at 06:41