13

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often the $d'$ (sensitivity index) is used for a similar purpose. The two are closely connected, and I believe they are equivalent to each other if certain assumptions are satisfied.

The $d'$ calculation is usually presented based on assuming normal distributions for the signal distributions (see wikipedia link above, for example). The ROC curve calculation does not make this assumption: it is applicable to any classifier that outputs a continuous-valued decision criterion that can be thresholded.

Wikipedia says that $d'$ is equivalent to $2 \text{AUC} - 1$. This seems correct if the assumptions of both are satisfied; but if the assumptions are not the same it's not a universal truth.

Is it fair to characterize the difference in assumptions as "AUC makes fewer assumptions about the underlying distributions"? Or is $d'$ actually just as widely applicable as AUC, but it's just common practice that people using $d'$ tend to use the calculation that assumes normal distributions? Are there any other differences in the underlying assumptions that I've missed?

Sycorax
  • 76,417
  • 20
  • 189
  • 313
Dan Stowell
  • 1,262
  • 1
  • 12
  • 22

1 Answers1

7

No. The maximum value of the AUC is 1. d' has no maximum.

I believe that d' is equal to the qnorm(AUC)*sqrt(2) (my memory of an old stats book I can't find right now but seems to check out against some data I found on the web). Here qnorm(x) is the "quantile function for the normal distribution" (R-speak). That is, it returns the value of the normal distribution for which x proportion of the distribution is below it.

Joel Lachter
  • 71
  • 1
  • 2
  • 2
    Thanks, this sounds correct to me - however, only if we assume the distributions are normal (because of the use of qnorm()). I've fixed the wikipedia wording. – Dan Stowell Aug 29 '15 at 08:25
  • The signal and noise distributions being normal is an assumption of the basic calculations of d'. – russellpierce Oct 07 '20 at 02:52