0

I have trained a ResNet (convolutional neural network) with a global average pooling layer at the end and a softmax layer with 3 units for the classification of 3 classes. As input to the network I have images with 3 color channels.

Now I would like to add one additional one-dimensional input (just one number per image). I see the possibility to add this input to the vector which I get from the global average pooling layer. The drawback is that this vector is then directly fed into the softmax layer.

Another possibility would be to append the additional input to the result of the average pooling layer but then adding some fully-connected layers (dense layers) after the global average pooling layer. Unfortunately, adding such fully-connected layers makes the performance much worse.

Are there any other possibilities to add this additional input or which one is the best way?

machinery
  • 1,474
  • 4
  • 18
  • 30
  • When you append the value to the output of the global average layer, you say that it goes directly to softmax. When you append the value and use some (presumably meaning 2 or more) FC layers, you say the performance is bad. What happens if you use a single FC layer before softmax? // In what sense is the performance worse? Lower training loss, lower validation accuracy, or something else? – Sycorax Apr 27 '21 at 13:46
  • @Sycorax If I use a single FC layer before softmax, performance is also dropping. That means lower validation accuracy. – machinery Apr 27 '21 at 21:15
  • When you say that the accuracy is dropping, can you [edit] to report the sample size and the accuracy before & after making the change? See also https://stats.stackexchange.com/questions/312780/why-is-accuracy-not-the-best-measure-for-assessing-classification-models – Sycorax Apr 27 '21 at 22:08
  • @Sycorax I did not use accuracy but (micro and macro averaged) AUC due to class imbalance. The sample size is around 30'000 samples (15000 class 1, 10000 class 2 and 5000 class 3). The accuracy before is around 0.8 and after it is around 0.6. – machinery May 02 '21 at 21:39
  • @Sycorax What configuratoin of the FC layer(s) before softmax would you recommend? Meaning number of units, layers and so on. – machinery May 06 '21 at 23:22

0 Answers0