6

We've all seen how $p$-values can be misinterpreted to draw false inferences. However, I'm interested in learning more about theoretical/mathematical/philosophical objections to the hypothesis-testing-paradigm, ie. I'd like to learn why someone would prefer (say) a likelihood ratio over a hypothesis test.

I've read some articles by Andrew Gelman on this topic, but I'm trying to find more literature. Any help would be greatly appreciated.

Stijn
  • 1,550
  • 1
  • 12
  • 20
  • 1
    Couple of books that you might be interested in "What if there were no significance tests?", edited by Harlow, Mulaik and Steiger, and "The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives" by Ziliak and McCloskey. – Jeremy Miles Mar 26 '13 at 15:14
  • See: http://stats.stackexchange.com/q/10510/3748 – Michael Bishop Mar 26 '13 at 19:57
  • This is just my own opinion, but too many people confuse statistical hypotheses with scientific ones. Even if you have "rejected the null hypothesis" that some statistical parameter = 0, it doesn't necessarily mean you've even asked a scientific question (from a philosophical, epistemological point of view). – D L Dahly Mar 26 '13 at 19:04

3 Answers3

3

This paper characterizes the publication/scientific practice in psychology as being a ritual of finding p<.05. They argue against hypothesis testing without solid theoretical basis. Note, though, that it is--if at all--more philosophical than anything else. But maybe it helps.

Nameless
  • 1,180
  • 11
  • 22
3

Well, try virtually any book about Bayesian statistics. I haven't find one that doesn't have at least a few paragraphs debunking practice of significance testing. Really. And Gelman's books are good example of that. I used to remember relevant titles (I still remember my first one: E.T. Jaynes "Probability theory - the logic of science". Jaynes definitely wrote with passion, and the controversy over significance testing is something, that really made me do statistics professionally ;-) ), but soon I got overwhelmed by the abundance of literature on this topic.

Try also the phrase +bayesian significance testing fisher in Google Scholar and I guarantee you will find at least dozen good references (and some interesting info too).

Adam Ryczkowski
  • 2,092
  • 2
  • 18
  • 33
1

I'm reading an article right now called "Principles of Inference and Their Consequences" by D.G. Mayo and M. Kruse. It's a good article so far about how hypothesis testing can violate principles like the likelihood principle(LP). They go through a concrete example of coin-tossing and show how the concept of statistical significance violates the LP.

Fly
  • 161
  • 4