As we have seen, Bayesian and Frequentist approaches sometimes give different answers to real world problem. So why can't we perform experiments and determine (with a very high likelyhood) who is right? Or is it the case that sometimes the Frequentist is right, sometimes the Bayesian?
Asked
Active
Viewed 111 times
2
-
2I'm by far not an expert but there may be one problem. Suppose you could test which model (or approach) is better, do you test it in Bayesian or Frequentist way? : ) – Łukasz Grad Mar 05 '17 at 17:17
-
@ŁukaszGrad Well the accepted answer in the linked question considers a coin being flipped 14 times. The coin has a probability p of coming up heads and it happens to come up heads 10/14 times. The question is, would you now bet on the coin coming up heads the next 2 times in a row? Bayesian and Frequentist approaches give different answers. We can now flip the coin 2 times, record who wins, and repeat this experiment many times. Is this biased towards Bayesianism or Frequantionism in any way? – Ovi Mar 05 '17 at 17:24
-
2They don't give different answers. They answer different questions. – jaradniemi Mar 05 '17 at 20:48
-
@jaradniemi I don't know any statistics, so I can't judge for myself. But then I have to ask, why does an accepted and upvoted answer to a question with 8k views seem to contradict that? – Ovi Mar 05 '17 at 22:27
-
@Xi'an I guess my definition of right is "the one which agrees most closely with repeated experiment". – Ovi Mar 06 '17 at 14:35
-
Repeated experiment under which conditions? If you keep the parameter constant, this is a frequentist framework. If you keep changing the parameter value, this is a Bayesian framework. – Xi'an Mar 06 '17 at 16:51
1 Answers
3
This is a most interesting question in that it does not have an answer. The major issue with statistics is that there is no "truth" in realistic settings and thus that reaching or uncovering this "truth" is impossible. Given a dataset, and an assumed model (that is most likely "wrong"), it is possible to describe the probabilistic properties of a statistical procedure with regard to this model. But depending on the procedure, the model, and the property, one procedure can be better or worse.
An example is provided by classical decision theory: let us assume an observation $x$ that is taken as a realisation of a Normal $\text{N}(\theta,1)$ model and let us settle on evaluating errors by a simple squared difference $(d-\theta)^2$ [all of those are assumptions and choices, there is nothing absolute in those!]. The error cannot be minimised uniformly over all values of $\theta$. If one settles for an average error, one can pick the frequentist risk $$R(\delta,\theta)=\int_\mathcal{X} (\delta(x)-\theta)^2 \varphi(x-\theta)\text{d}x$$ or the Bayesian risk $$r(\delta,\pi) = \int_\Theta (\delta(x)-\theta)^2 \pi(\theta|x)\text{d}\theta$$ the later allowing for an optimal solution that depends on the prior $\pi$. While the former does not allow for an optimal solution for all values of $\theta$. One must then consider minimaxity or admissibility to sift though procedures. And even then there may be several minimax or several admissible procedures, most of which will be Bayesian.
You could take a look at the discussion of another question: Concrete-examples-of-a-frequentist-approach-that-is-superior-to-a-Bayesian-one
