If I want to use naive bayes with laplace smoothing and therefore add 1 to probabilities with the value of 0, what does this mean for probabilities which have the actual value of 1?
Asked
Active
Viewed 7,845 times
1
-
It is not clear what you are asking. – Michael R. Chernick Apr 09 '17 at 22:46
-
When using laplace smoothing with naive bayes, do I have to add the value 1 to all my probabilities, or just probabilities with the value of 0? – link Apr 09 '17 at 22:50
-
Yes I read your question but it doesn't make sense as written. – Michael R. Chernick Apr 09 '17 at 22:52
-
link ; perhaps the value is added to zero cells of counts (or all cells?) which are then used to estimate the probabilities, rather than adding a value to the probabilities themselves – user20650 Apr 09 '17 at 22:59
-
perhaps of interest http://stats.stackexchange.com/questions/108797/in-naive-bayes-why-bother-with-laplacian-smoothing-when-we-have-unknown-words-i – user20650 Apr 09 '17 at 23:03
1 Answers
2
A value 1 is added to each feature count (not just to the feature having a count/frequency of 0 ). To be more specific, consider the case of text classification. Let us suppose that $Y_k, k=1,2,\cdots K$ denote the labels of the $K$ classes, $X_j$ denote the $j^{th}$ word and $V$ denote the total number of distinct words(vocabulary size) in all of the $n=\sum_{k=1}^{K}n_{k}$ documents, where $n_{k}$ denote the number of documents labelled as $Y_{k}$. Then, the Laplace estimate of the probability for the word $X_{j}$ in the class $Y_{k}$ is given by $$P(X_{j}|Y_{k})\dfrac{Count(X_{j},Y_{k})+1}{\sum_{j}^{V}(Count(X_{j},Y_{k})+1)}.$$
L.V.Rao
- 1,909
- 15
- 23