1

Let $(X,Y): \Omega \to (\mathbb{R}^d, F)$, with $F = \left\{ 1, \dots, n \right\}$ be a random variable. According to Wikipedia the Bayes classifier is $$C_B(x) = \arg\, \max \mathbb{P}(Y= i \,|\, X=x),$$ with $x \in \mathbb{R}^d$. Now I understand the idea, "pick the most likely one". However, if $X$ has a density, then $X=x$ is a zero probability event. How should conditioning on this event be understood?

Of course you can define something like $$\mathbb{P}(Y=i \,|\, X=x) = \lim \limits_{\varepsilon \to 0} \mathbb{P}(Y=i \, | \, X \in B(x, \varepsilon)),$$ where $B(x,\varepsilon)$ is a ball of radius $\varepsilon > 0$ around $x$, but I'm unable to find a reference for this.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Matias Heikkilä
  • 141
  • 4
  • 1
    While $\mathbb{P}(X=x)=0$, the conditional distribution of $Y$ given $X=x$ is uniquely defined almost everywhere. – Xi'an Nov 15 '16 at 13:37
  • 2
    $P(Y=i \vert X=x)$ is to be interpreted as having observed $x$, what is the probability that it comes from class $i$. It is computed from $f(x\vert Y=i)p(Y=i)$, where we assume that the class conditional densities $f(x\vert Y=i)$ and prior probs $p(Y=i)$ are known in advance. – L.V.Rao Nov 15 '16 at 14:02
  • Recent related question: http://stats.stackexchange.com/questions/246009/how-to-interpret-mathbbpba-when-mathbbpa-0 – Tim Nov 15 '16 at 14:57

0 Answers0