1

Although I am completely new to Bayesian Analysis I struggle sometimes when trying to investigate some intersections between Bayesian and Frequentist analysis. I would like to discuss the different implications based on the following simple example:

Let $y=(y_1,\ldots,y_N)$ were $y_i|\mu \sim N(\mu,1)$, $\mu\in\mathbb{R}$. Hereby $\mu$ is not known to us. A Frequentist would compute $\hat{\mu}=\frac{1}{N}\sum_{i=1}^{N}y_i$. For the estimator $\hat{\mu}$ it holds that $\hat{\mu} \sim N(\mu,\frac{1}{N})$.

In a Bayesian approach we could assign a normal prior for $\mu$ with infinite variance $\mu\sim N(\mu_0,\lambda^{-1}), \lambda\rightarrow \infty$ and obtain a normal distributed posterior density $\mu|y\sim N(\hat{\mu},\frac{1}{N})$.

I am wondering how $\hat{\mu} \sim N(\mu,\frac{1}{N})$ and $\mu|y\sim N(\hat{\mu},\frac{1}{N})$ can be interpreted and what different implications you can draw out of those formulas. Is it a correct Bayesian interpretation to state that after observing our data $y$ the variance term $\frac{1}{N}$ reflects the 'uncertainty' that is incorporated in choosing $\hat{\mu}$ as our decision rule? On the other hand side what does the Frequentist approach tells us regarding the 'true' parameter? Each and every comment is welcome!

muffin1974
  • 1,152
  • 11
  • 24
  • special case of http://stats.stackexchange.com/questions/31867/bayesian-vs-frequentist-interpretations-of-probability?rq=1 ? – Christoph Hanck Jun 19 '15 at 10:40
  • Surely this question is related to the general discussion about interpreting Bayesian vs. Frequentist probabilites. However, I do struggle in transferring the philosophical considerations into an example like the one given above. I would love to see what the Bayesian interpretation of uncertainty and the Frequentist idea of asymptotic behaviour implies for these two representations of $\hat{\mu}$ and $\mu|y$. – muffin1974 Jun 19 '15 at 11:05

1 Answers1

4

So, I welcome any comments or corrections, it's been a while since I've sat in front of a textbook.

Insofar as I've always thought of it, the frequentist isn't as interested in the distribution of $\hat{\mu}$ as the Bayesian is of $\mu \mid y$ (I'm totally going to get flamed for that statement). Why do I say this?

  • In the frequentist view of things, $\mu$ is a degenerate random variable, and $\hat{\mu}$ is our best guess at $\mu$. Hence, we may use the (implicit) distribution of the estimator as a means of handling the error of our guess, but we can't really interpret $$P( \hat{\mu} \in [a,b]) = p(a,b)$$ as something like

    The probability that $\mu$ lies in $[a,b]$ is $p(a,b)$.

    Instead, $p(a,b)$ encodes our uncertainty in $\hat{\mu}$ and -- this is my opinion -- this doesn't really help us get a handle on $\mu$...

  • In the bayesian view, however (as you've identified), $\mu$ is a (nondegenerate) random variable, and our collection of data, $y$, affords us the ability to say things like $$P(\mu \in [a,b] \mid y) = p(a,b)$$

    The probability that $\mu$ lies in $[a,b]$ is $p(a,b)$.

... I'm not sure if this helps, but leave a comment and I'll cleanup/elaborate as necessary!
StevieP
  • 508
  • 4
  • 10
  • Thank you for your great answer.I think as $\mu$ is unknown the amount of information we obtain regarding $\mu$ should be the same for both of the Probability Interpretations and therefore $P(\hat{\mu} \in [a,b])=p(a,b)$ does not really help to learn something about $\mu$ that is not incorporated in $\mu|y$. For me the representation $\hat{\mu}\sim N(\mu,\frac{1}{N})$ just states our observed variable $\hat{\mu}$ is one realization of a Normal variable with unknown mean. Therefore we know nothing about $\mu$. But what is then the value of the distribution of $\hat{\mu}$ for the Frequentist? – muffin1974 Jun 19 '15 at 11:25
  • 1
    So, let me start off backwards: _for me_ that $\hat{\mu}$'s distribution is specified relative to an unknown quantity makes it pretty unhelpful _if I'm trying to do inference on_ $\mu$. Maybe other people have less obvious things they want to infer... Me, on the other hand, I'm a simple guy. If I'm trying to estimate $\mu$, I'd like some easily interpret-able statements about it; sadly, frequentist confidence intervals are hard for non-statisticians to interpret... – StevieP Jun 22 '15 at 21:34
  • 1
    Towards your 3rd sentence, however: I'm not sure how $P(\hat{\mu} \in [a,b])$ is incorporated in $\mu \mid y$... Sure, $\mu \mid y \sim N(\hat{\mu}, N^{-1})$, but I'm not seeing an obvious connection aside from the fact that both mathematical statements reference $\hat{\mu}$. – StevieP Jun 22 '15 at 21:36