Discrepancy between categorical cross entropy and classification accuracy

Question

I have a convolution neural network with random weights initialized and Trained to perform binary classification. I have 2000 images as training data and 2000 validation data. The problem I am trying to solve is if the image is healthy or not.The loss function used is categorical cross entropy.

I have implemented the model in tensorflow.(tf.nn.sparse_softmax_cross_entropy_with_logits) The validation loss for that model with random weights is 0.71 and an accuracy was 58%. However, the same architecture as above initialized with a pre-trained weights gives a loss of 0.79 and accuracy 65% (Both the models converged without overfitting)

Does any one have any idea on why the loss in the second case is high yet has a better accuracy compared to the first one?

To put it another way, what is the contributing factor to the discrepancy between classification accuracy and categorical cross entropy, where a higher cross entropy corresponds to a better classification accuracy?

The learning curves for the both the cases are shown.

I think that if you want people to give a reasonable explanation you need to supply more details. — Michael R. Chernick, Apr 30 '17 at 16:28
I edited my question. Do you suggest adding more information? — Akshata Bhat, Apr 30 '17 at 16:38
What variables are you using? What is the real problem you are trying to address? How much data do you have? How do you know that you are not overfitting in either case? — Michael R. Chernick, Apr 30 '17 at 16:52
I have added the learning curves and a better description. I would be happy to hear your suggestions — Akshata Bhat, Apr 30 '17 at 17:25
What do you mean by pre-trained weights in this case? Where are the non-random initialized matrices coming from? — user2879934, Apr 30 '17 at 17:41
random initialization is -- Xavier weight initialization and pretrained weights come from a unsupervised deep learning approach. — Akshata Bhat, Apr 30 '17 at 17:42
I am using an unsupervised deep learning algorithm, were in I make use of a lot of unlabelled data to learn weights and initialise them to my supervised network. Kind of semisupervised learning. — Akshata Bhat, Apr 30 '17 at 17:45
Well usually, for binary classification, you would use sigmoid output nodes, otherwise you have to one-hot encode for cat-cross-entropy...that is my only suggestion — user2879934, Apr 30 '17 at 17:46
Yes all that I took care off. But my loss is higher in case of the pretrained weights, but it has a better accuracy — Akshata Bhat, Apr 30 '17 at 17:48

Discrepancy between categorical cross entropy and classification accuracy

0 Answers0

Linked