1

I am going through this post in trying to prove the probabilistic interpretation of the AUC for a ROC Curve (for a classifier):

The AUC for a ROC curve is the the probability of the classifier scoring a randomly chosen positive point higher than a randomly chosen negative one.

I was able to follow till the end, but am a little stuck on getting the final result. So far I have the following

\begin{align*} {AUC} &= \int_0^1 tpr(fpr) d(fpr)\\ &= \int_0^1 tpr(fpr(s))d(fpr(s))\\ &= \int_{+\infty}^{-\infty} tpr(s)fpr'(s)ds\\ &= \int_{+\infty}^{-\infty} \left(1-F_1(s)\right) \left(-f_{-1}(s)\right)ds\\ &= \int_{-\infty}^{+\infty} (1-F_1(s)) (f_{-1}(s))ds \\ &= \int_{-\infty}^{+\infty} Pr(s<X_{1}) (f_{-1}(s))ds \\ &= \int_{-\infty}^{+\infty} Pr(X_{-1}<X_{1}\ |\ X_{-1}=s) (f_{-1}(s))ds \\ &= \int_{-\infty}^{+\infty} Pr(X_{-1}<X_{1}, X_{-1}=s ) ds \\ &= Pr(X_{-1}<X_{1} ). % &= \int_{-\infty}^{+\infty} P(S>s) (f_{-1}(s))ds \\ \end{align*}

I am struggling in my confidence that

$$\int_{-\infty}^{+\infty} Pr(X_{-1}<X_{1}\ |\ X_{-1}=s) (f_{-1}(s))ds = \int_{-\infty}^{+\infty} Pr(X_{-1}<X_{1}, X_{-1}=s ) ds$$

is correct. I found the result in this post, but cannot find it elsewhere for continuous random variables. Is this a standard definition without proof? I have not seen it before and I can't find anything about conditional probabilities that makes me get it, could I get some help on that part?

Thanks

sadlyfe
  • 175
  • 1
  • 7
  • should the integral of title from $-\infty$ to $\infty$, but not 0 to 1? – Tan Jan 06 '21 at 04:07
  • Related references: De Schuymer, Bart, Hans De Meyer, and Bernard De Baets. "A fuzzy approach to stochastic dominance of random variables", in _International Fuzzy Systems Association World Congress_ 2003; Montes Gutiérrez, I., "Comparison of alternatives under uncertainty and imprecision", doctoral thesis, Universidad de Oviedo, 2014. – Peter O. Jan 06 '21 at 04:49

2 Answers2

4

It's easier to see if everything is discrete, so you have on the left

$$\sum_{s=-\infty}^\infty P[X_{-1}<X_1 | X_{-1}=s] P[X_{-1}=s]$$ and on the right $$\sum_{s=-\infty}^\infty P[X_{-1}<X_1 \cap X_{-1}=s] $$ so for each $s$ it's just the equality $P(A|B)P(B)=P(A\cap B)$

In the continuous case, probabilities with equalities turn into densities times $ds$, and sums into integrals.

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
1

It's probably easiest to use the law of iterated expectations. To do so we can write the probability as the expectation of an indicator: $$ \mathbb{P}(X_1 > X_2) = \mathbb{E}[ \mathbb{I}\{X_1 > X_2 \} ]$$ The law of iterated expectations states that \begin{align*} \mathbb{E}[ \mathbb{I}\{X_1 > X_2 \} ] &= \mathbb{E}[ \mathbb{E}[\mathbb{I}\{X_1 > X_2 \} \mid X_2]] \\ &= \mathbb{E}[ \mathbb{P}(X_1 > X_2 \mid X_2)] \end{align*} The second line substitutes the definition of conditional probability as the expectation of an indicator function (similar to what we did before).

Finally, if $X_2$ has a continuous distribution, we can put the results together to show that $$ \mathbb{P}(X_1 > X_2) = \mathbb{E}[ \mathbb{P}(X_1 > X_2 \mid X_2)] = \int \mathbb{P}(X_1 > X_2 \mid X_2 = x_2) f(x_2) dx_2 $$ Continuity of $X_2$ is only used for the second equality, which substitutes the integral definition of the expectation. I like this proof because you don't have to take a stance on whether $X_1$ or $X_2$ are discrete/continuous until the last step.

ecnmetrician
  • 602
  • 3
  • 8