1

When we draw a ROC curve, how can we determine the optimal threshold? I think, the point that is closest to (0,1) vector is the best threshold. However, if we cannot calculate the distance, how can we determine the best threshold? Is there any other way to know that?

Also, I think eyeballing is a pretty good way. How do you guys think about the eyeballing method?

QWEQWE
  • 471
  • 1
  • 9
  • There are quite a few related questions that are likely duplicates (see the Related links on the right hand side). If you have fixed costs you can tie to to the confusion matrix (costs/benefits for different cells) you can determine an optimal cut point. If not, 'optimal' is not well defined. – Andy W Jun 13 '21 at 15:42

1 Answers1

4

In an ROC analysis, any given threshold for the continuous variable will, in the sample, produce some pair of sensitivity and (1 - specificity) values. Which pair is best is an extra-statistical decision. It depends on the costs of false positives and the benefits of true positives as perceived or assessed for the application or topic in question. An example: “It is better that ten guilty people go free than to convict one innocent person.”

rolando2
  • 11,645
  • 1
  • 39
  • 60
  • Yes, the question always is: "optimal for what?" (+1) The formulas relating costs and benefits for a well calibrated binary model to the probability cutoff are provided [here](https://stats.stackexchange.com/a/502004/28500). – EdM Jun 13 '21 at 21:22