How to normalize the output of a neural network

Question

We have a VGG16 network trained from scratch with a Sigmoid output function. We have 6 classes and the usual output looks like this:

scores': [6.494849458249519e-08, 1.8738395510808914e-06, 3.010111981893715e-07, 0.0, 0.0, 0.8633317947387695]

The problem is that the output value is very low in each class, I would like to have a normalized output that sums to 1.0 Thanks

2

Don't you have softmax layer? – gunes Mar 09 '20 at 12:32

Sycorax · Accepted Answer · 2020-03-10T18:02:31.397

1

Sigmoid outputs will each vary between 0 and 1, but if you have $k$ sigmoid units, then the total can vary between 0 and $k$. By contrast, a softmax function sums to 1 and has non-negative values.

edited Mar 10 '20 at 18:02

answered Mar 09 '20 at 12:45

Sycorax

76,417
20
189
313

score 0 · Answer 2 · edited Mar 09 '20 at 15:11

0

If you are concerned about the output being too low, try re-scaling the output. I don't clearly understand what you mean by normed output sum to 1.

However, if you want the sum of output to be 1, one possible way is considering log-likelihood or log-it transformation. The log function takes into account properly the negative indices in powers of 10.

edited Mar 09 '20 at 15:11

Roger Vadim

1,481
6
17

answered Mar 09 '20 at 13:05

Deeptendu Santra

1
2

How to normalize the output of a neural network

2 Answers2