9

How did academics support hypotheses before the null hypothesis significance testing (NHST) framework was, in part, introduced and democratized by Fisher/Neyman & Pearson? Suppose NHST was never a thing, what are some plausible frameworks academics could employ to support their hypotheses today? Are there alternatives based on mathematics outside of statistics and/or probability?

user
  • 2,010
  • 8
  • 23
  • A little like asking how we would get along if no one had invented multiplication; not sure to overcome that. Maybe confidence intervals could substitute for some tests of hypotheses. – BruceET Sep 21 '20 at 05:56
  • 3
    There were many tests performed before either Fisher or Neyman and Pearson. The earliest hypothesis test was probably Arbuthnot (1710), who essentially performed a sign test (binomial test with $p_0=\frac12$). However, in any case science mostly managed pretty well without statistical hypothesis tests. Careful experiment, observation and repeated replication can render conclusions evident enough. – Glen_b Sep 21 '20 at 09:31

1 Answers1

13

Neither Fisher nor Neyman and Pearson proposed a "null hypothesis significance testing framework". Instead, Fisher demonstrated the significance testing framework and Neyman and Pearson later demonstrated the hypothesis testing framework. They are not the same and they are not similar in their objectives. The significance testing framework attempts to quantify the evidence in the data against a null hypothesis, and uses a continuous p-value. The hypothesis testing procedure entails a decision to reject or not reject the null hypothesis and it does not use a p-value.

The NHST hybrid that you ask about is an incoherent mixture of two incompatible approaches. Neither Fisher nor Neyman and Pearson would be happy to have their names attached.

Please see this paper for a more complete explanation: https://link.springer.com/chapter/10.1007/164_2019_286

Michael Lew
  • 10,995
  • 2
  • 29
  • 47
  • 1
    Another useful reference (to Aris Spanos' textbook chapter on the matter) is included in [this answer](https://stats.stackexchange.com/questions/112769/is-the-hybrid-between-fisher-and-neyman-pearson-approaches-to-statistical-test/112786#112786) by Alecos Papadopoulos. – Richard Hardy Sep 21 '20 at 07:15
  • Thank you for pointing out the distinction between the two frameworks; the paper was a good read. – user Sep 21 '20 at 18:20