0

I am trying to write down the exact definition of a bayesian statistical model in a similar way as the definition for a statistical model.

So far I have the definition:

A statistical model is a pair $(X,(\mathbb{P}_{\theta})_{\theta\in\Theta})$, where $(\mathbb{P}_{\theta})_{\theta\in\Theta}$ is a family of probability distributions on a measurable space $(S,\mathcal{A})$. $X$ is a random variable mapping to $S$, whose distribution has a free parameter $\theta \in \Theta$. This means, we know a function $\theta \mapsto \rho_{\theta}$ with $$ P_{\theta}(X \in da) = \rho_{\theta}(a)da $$ \begin{align*} \text{with}&&P_{\theta}(X \in da) & := P(X=a) \text{, for }S\text{ discrete and} \\ &&P_{\theta}(X \in da) &:= f(a)da \text{, for density }f(a). \end{align*}

Now I wanted to add that in bayesian models the parameter $\theta$ is considered a random variable on its own:

Let $(S\times\Theta,\mathcal{A}\times\mathcal{B},P)$, with an appropriate $\mathcal{B}$, be a probability space and let $(X,T)$ be a random variable mapping to $(S\times\Theta)$. For the family $(\mathbb{P}_\theta)_{\theta\in\Theta}$ of conditional distributions $$\mathbb{P}_{\theta}(\cdot):=P(\cdot|T=\theta)$$ and the marginal distribution $$ \Pi:\mathcal{B}\rightarrow[0,1],\quad \Pi(B) := P(T\in B)= P((X,T)\in S\times B),$$ the statistical model $(X,\mathbb{P}_{\theta})$ is called a bayesian statistical model with A-Priori-distribution $\Pi$.

My problem/question is, that I know $\mathbb{P}_{\theta}(\cdot):=P(\cdot|T=\theta)$ is wrong, because it must be $$P(\cdot|T\in B), \quad B\subset \Theta$$ for it to be a conditional distribution because $\{T=\theta\}$ would be of measure zero if $\Theta$ is not discrete.

However, I am not sure whether the definition of $\mathbb{P}_{\theta}(\cdot)$ then still makes sense as in the end, the 'true parameter' which we want to find is a single value $\theta_0$ which produces the observations $X_1,...,X_n\sim\mathbb{P}_{\theta_0}$ ? I.o.w. how can I reconcile $\mathbb{P}_{\theta}\text{ and }P(\cdot|T\in B)\text{ ?}$

Thank you very much in advance!

klm
  • 63
  • 4
  • 3
    I don't this is a question about Bayesian statistics as much as it is a question about conditional distributions. To make the question more succinct, you could ask, why, in a bivariate normal distribution $p(x,y)$ of $(X,Y)$, can we speak of the conditional distribution $p(y|X=x)$, considering that $X=x$ is an event with probability zero? – BigBendRegion Jul 13 '21 at 13:11
  • 1
    Perhaps https://stats.stackexchange.com/questions/230545/intuition-for-conditional-expectation-of-sigma-algebra addresses the issue? For more like this, use [this site search](https://stats.stackexchange.com/search?q=sigma+algebra+condition*). – whuber Jul 13 '21 at 13:48
  • 1
    @whuber Thank you very much! I didn't know yet that conditioning on an event implicitly means conditioning on the sigma algebra generated by it, nor did I even know about conditioning on sigma algebras, so I will definitely look into it! – klm Jul 13 '21 at 14:10

0 Answers0