0

I am using the epoch validation accuracy as a metric to save the model current status if there is a progress but I was wondering if it is the right way or should I use the lowest epoch validation loss or a combination of both ? I am working on multi class classification problem

ou2105
  • 113
  • 4
  • 1
    You should measure the thing that you care about. Accuracy specifically has a number of downsides; see https://stats.stackexchange.com/questions/312780/why-is-accuracy-not-the-best-measure-for-assessing-classification-models/312787#312787 – Sycorax Mar 02 '22 at 01:47
  • most lickely I care about accuracy and its related forms top2,top3,top4,top5 and also kappa and f1 measure – ou2105 Mar 02 '22 at 01:48
  • Sounds like you have your answer. What’s giving you doubt? – Sycorax Mar 02 '22 at 01:55
  • a combination of factors the fact that there is a huge gap between my validation accuracy and training one 65% and 95% , the fact that my cross entropy loss is very high in validation superior then 1 and small in training less then 0.1, and the fact that my validation loss and accuracy cuvres doesn't what they should look like , where as the training loss and accuracy curves looks good – ou2105 Mar 02 '22 at 02:08
  • Sounds like [tag:overfitting]. See https://stats.stackexchange.com/questions/365778/what-should-i-do-when-my-neural-network-doesnt-generalize-well for suggestions. – Sycorax Mar 02 '22 at 02:10
  • that is what I expected to. PS I have a a multi label classificaion problem with 12 classes with a training set of 5797 , validation set of 889 and a test set of 1457 for I am using a simple cnn I was thinking that I have very few samples for such a task, – ou2105 Mar 02 '22 at 02:15
  • Sycorax I taught that the loss is how the model performs well on the data in some how this definitions intuively imply that this is what anyone should care about no matters what ? – ou2105 Mar 02 '22 at 04:05

0 Answers0