3

Wikipedia states (without citation):

Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests.

What are examples of exact tests which are not permutation tests? I assume that Fisher's exact test is also considered to be a permutation test since it looks to my understanding at all possible permutations of contingency tables (see also here). Is this correct?

Arne Jonas Warnke
  • 3,085
  • 1
  • 22
  • 40

1 Answers1

2

Any test that keeps the Type I error rate of the test at the specified level is exact. In this sense, for example, the well known one-sample t test for normally distributed data is exact. Similarly, the two-sample t test for normally distributed data of equal variance is exact.

Actually, most of the "standard" tests are exact on their more restrictive formulations.

peuhp
  • 4,622
  • 20
  • 38
  • I'm not sure that this is correct. I thought an 'exact' test is one where the assumptions do not include any approximations, such as an assumption that the data or errors are normally distributed. That is quite different from simply providing specified alpha under an assumption that can only be approximately true. – Michael Lew Dec 20 '19 at 19:58