0

I have been working with binary classification problem, were prediction of both my classes were completely/equally important.

  • data balance ratio - 46%(0) & 54 % (1)
  • Problem statement : Predicting whether the product will Pass or Fail.

My question would be, which evaluation metrics should I choose to evaluate my Model ? Is there is any specific rule to use Precision, Recall, F1 score, based on the type of problem ?

Currently I am working only with Accuracy to test my models.

Kindly share your thoughts.

Mari
  • 111
  • 5
  • 1
    When you say "completely important", do you mean "equally important", i.e. the cost of a false positive prediction compared to a true negative is equal to the cost of a false negative prediction compared to a true positive? If so, maximising accuracy will minimise cost. – Henry Jun 08 '20 at 07:48
  • Yes, both of my classes is of equal Importance. So based on your comment, can I take accuracy as an evaluation metrics ? – Mari Jun 08 '20 at 07:57
  • 1
    If there is little structure in your data, then optimizing accuracy may lead to your model classifying *everything* as "1". Yes, this will minimize expected cost. It may not be what the user expects, though. I very strongly recommend [Why is accuracy not the best measure for assessing classification models?](https://stats.stackexchange.com/q/312780/1352), the answers suggest some alternatives to accuracy. I personally recommend proper scoring rules to optimize probabilistic classifications, followed by a long talk with the user about subsequent decisions. – Stephan Kolassa Jun 08 '20 at 08:19

0 Answers0