ROC AUC score significance

Question

I used the sklearn.metric roc_auc_score,it gave me a value 0.91.

What is the does this number mean?

I am interested to learn how this is calculated,could someone please direct me to some information on this?

https://de.wikipedia.org/wiki/Receiver_Operating_Characteristic — Nikolas Rieble, Dec 20 '16 at 09:04
and also: http://stats.stackexchange.com/questions/132777/what-does-auc-stand-for-and-what-is-it/132832#132832 — Nikolas Rieble, Dec 20 '16 at 09:05

score 1 · Answer 1 · answered Dec 20 '16 at 09:08

1

I think this is covered in Wikipedia and many other similar posts here. Please look at the comments.

AUC of 0.91 is much better than a random model (AUC=0.5), but it doesn't mean your model is good. You will need to compare your model with a reference model. If your reference model has AUC 0.95, your AUC 0.91 is bad. However, if your reference AUC is 0.70, then your AUC 0.91 is good.

answered Dec 20 '16 at 09:08

SmallChess

6,764
4
27
48

How would you choose a reference model? – Nikolas Rieble Dec 20 '16 at 09:14
@NikolasRieble That depends on your problem. Specificity, sensitivity? – SmallChess Dec 20 '16 at 09:47
By a reference model do you simply mean another measurement of accuracy/ quality (Specificity, sensitivity)? Or do you mean actually another modell (such as SVM) to compare both performances? – Nikolas Rieble Dec 20 '16 at 09:51

ROC AUC score significance

1 Answers1