2

If the accuracy of $classifier1$ is statistically significantly better than $classifier2$ as per some hypothesis test, and likewise the accuracy of $classifier2$ is statistically significantly better than $classifier3$, does it follow that $classifier1$ is statistically significantly better than $classifier3$. This question relates to my question asked here.

Community
  • 1
entropy
  • 1,122
  • 2
  • 12
  • 21
  • 3
    I believe you made a typographical error and wish to ask (in shorthand notation) whether $c_1 \ge c_2 \ge c_3$ implies $c_1 \ge c_3$. The answer depends on what you mean by "statistically better." Be careful how you answer that!--a natural meaning of "better" in this context is closely related to [Efron's nontransitive dice](http://en.wikipedia.org/wiki/Nontransitive_dice). – whuber Dec 17 '12 at 19:59
  • I reviewed the question. – entropy Dec 17 '12 at 20:29
  • You still have that typo. After all, if $c_1$ is better than $c_2$ and $c_3$ also is better than $c_2$, you still have no information to compare $c_1$ to $c_3$: you only know there is something worse than both. – whuber Dec 17 '12 at 21:54
  • blond moment. yes. – entropy Dec 17 '12 at 22:18
  • by statistically better I mean the accuracy of one is better than the other by some hypothesis test. – entropy Jan 02 '13 at 22:49
  • OK, but you need to be more specific: what data would you be using for the tests? (The correct answer might depend on whether you use independent sets of data for the pairwise tests or a common set of data for all pairwise tests, or whether you conduct a simultaneous test of all classifiers.) – whuber Jan 03 '13 at 15:28
  • Accuracy of each classifier estimated by cross validation on same same dataset. – entropy Jan 04 '13 at 23:08

0 Answers0