I want to draw my own ROC curve, such as explained in the accepted answer to this question:
However, it isn't clearly explained what the 'score' used represents. It is used to reorder the rows by their probabilities in descending order.
Does the probability score used (and, as shown in that answer) represent:
According to this website:
http://blog.revolutionanalytics.com/2016/11/calculating-auc.html
It is the positive class probability which should be used.
Calculating AUC: the area under a ROC Curve
by Bob Horton, Microsoft Senior Data Scientist
Receiver Operating Characteristic (ROC) curves are a popular way to visualize the tradeoffs between sensitivitiy and specificity in a binary classifier. In an earlier post, I described a simple “turtle’s eye view” of these plots: a classifier is used to sort cases in order from most to least likely to be positive, and a Logo-like turtle marches along this string of cases. The turtle considers all the cases it has passed as having tested positive. Depending on their actual class they are either false positives (FP) or true positives (TP); this is equivalent to adjusting a score threshold. When the turtle passes a TP it takes a step upward on the y-axis, and when it passes a FP it takes a step rightward on the x-axis. The step sizes are inversely proportional to the number of actual positives (in the y-direction) or negatives (in the x-direction), so the path always ends at coordinates (1, 1). The result is a plot of true positive rate (TPR, or specificity) against false positive rate (FPR, or 1 - sensitivity), which is all an ROC curve is."
