0

In the perfect explanation of Bayes' Theorem here as far as I know features of the class should be independent. The question is how to prove statistically that two given features are independent? I know Chi-test and Principal Component Analysis so far for analysing independence of two given variables. Is it sufficient to analyse with Chi-test independence of variables?

Note that the easy way to say that two variables are independent is to see their correlation, but sometimes it is not sufficient to say that two variables are not independent when there is no correlation between them.

Hope I could explain my question.

Community
  • 1
Bakhtiyor
  • 133
  • 2
  • 6

1 Answers1

2

When using naive bayes usually we don't bother to test for independence of the variables. This is the "naive" part of naive bayes. The algorithm works well for classification even if the variables are not independent.

If you did want to check for independence you would need an exponential amount of data with respect to the number of variables. It is unlikely that you would have enough data. Lack of correlation does not imply independence.

Aaron
  • 3,025
  • 14
  • 24
  • But what about this statement - In machine learning, naive Bayes classifiers are a family of simple probabilistic classifiers based on applying Bayes' theorem with strong (naive) independence assumptions between the features. Source: http://en.wikipedia.org/wiki/Naive_Bayes_classifier – Bakhtiyor Sep 28 '14 at 23:27
  • 1
    The assumption is that the variables are independence. It is naive because we know that the variables are likely not independent but we make the assumption anyway. – Aaron Sep 29 '14 at 00:51