My binary variable is whether a mortgage application is denied(1) or approved (0). Let's say I have two classifiers. One with AUROC = 0.75 and the other with AUROC = 0.85. Is it correct to state for example for the second classifier the following? The AUROC value of 0.85 implies that there is an 85% probability that a randomly selected denied observation is ranked higher than a randomly selected approved observation? How is ranked exactly to understand? Does it simply represent the probability that I will classify it as such (here 1)?
Asked
Active
Viewed 112 times
1
-
Should be helpful: http://stats.stackexchange.com/questions/132777/what-does-auc-stand-for-and-what-is-it/ – Alexey Grigorev Jan 02 '16 at 11:12
2 Answers
2
You are correct. The concordance probability ($c$-index; AUROC) is the probability that a randomly selected subject who had an event has a higher predicted probability than a randomly selected subject who did not have the event.
But note that you are not using a classifier if you are dealing with risk estimation.
The $c$-index is not a great measure for comparing two predictive models because it is not sensitive enough for that purpose. One model might have $c=0.84$ and the other $c=0.845$ yet the second be meaningfully better than the first.
Frank Harrell
- 74,029
- 5
- 148
- 322
-1
The AUROC value of 0.85 implies that a randomly selected denied observation has a precision of 0.85.
SamParker
- 74
- 4