1

Because of No Free Lunch Theorem, no machine learning algorithm can be said to be better than the other. However, in practice some algorithms are almost always better than the others. For example random forests and other tree based ensemble methods almost always beat other algorithms, see this paper.

One reply to this seemingly conflicting situation might be to note that No Free Lunch Theorem is true when we test algorithms on all possible data generating distributions. But in real life we have only a subset of these distributions. So my question is what are the properties of these real life probability distributions?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Sanyo Mn
  • 799
  • 6
  • 15

1 Answers1

3

The reasons that neural networks are such a hot topic right now is partially because of NN successes in image tasks, such as object recognition, facial recognition and other tasks. Trees simply don't do these tasks very well.

So, if you're willing to accept that images of ordinary objects are one type of data distribution, then you'll find that trees do not do very well against this distribution. Therefore, there is no contradiction with the NFL because another method (CNNs) outperforms trees.

See: What are the current state-of-the-art convolutional neural networks?

Sycorax
  • 76,417
  • 20
  • 189
  • 313