4

With the pROC package, I can do this:

true <- c(1, 1, 1, 0)
predicted <- c(0.5, 0.1, 0.6, 0.1)
roc(true, predicted)

which gives as expected:

> Call:
> roc.default(response = true, predictor = predicted)

> Data: predicted in 1 controls (true 0) < 3 cases (true 1).
> Area under the curve: 0.8333

I'm not sure why the following is working.

true <- c(1, 1, 1, 0)
predicted <- c(5, -1, 600, 10)
roc(true, predicted)

> Call:
> roc.default(response = true, predictor = predicted)

> Data: predicted in 1 controls (true 0) > 3 cases (true 1).
> Area under the curve: 0.6667

I was expecting an error about predictors being non-probability values, but things are silently accepted. Can someone tell me what's happening behind the scene?

elbord77
  • 205
  • 1
  • 8

2 Answers2

4

If you can rank your scores, you can use ROC analysis. Ranking can be done with real numbers, not only probabilities.

One of the interpretations of ROC AUC is the probability that a randomly-chosen positive is ranked higher than a randomly chosen negative; again, this ranking just requires that you can sort the data, but does not require that the data are assigned probabilities.

More elaboration on curves and can be found in our archives.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
3

To insist on Sycorax answer: as soon as you can rank your scores, you can build a ROC curve. The scores don't even have to be numbers. R has a data type called ordered which is a special type of factor with a built-in order. You can use it to build a ROC curve:

> true <- c(1, 1, 1, 0)
> predicted <- ordered(c("medium", "low", "high", "low"), levels=c("low", "medium", "high"))
> print(predicted)
[1] medium low    high   low   
Levels: low < medium < high
> rank(predicted)
[1] 3.0 1.5 4.0 1.5

> roc(true, predicted)

Call:
roc.default(response = true, predictor = predicted)

Data: predicted in 1 controls (true 0) < 3 cases (true 1).
Area under the curve: 0.8333
roc(true, predicted)
Calimo
  • 2,829
  • 17
  • 26