1

I have a dynamic range of 256x256 matrices. I want to have a CNN based binary classifier. The matrices are images with a very wide range of intensities (10 orders of magnitude).

I am afraid to use Mean Normalization or Min-Max Scaling, because it will introduce negative values into the feature space:

$x- \bar{x} \over \sigma$ or $\frac{x - \bar{x}}{\text{max}(x)-\text{min}(x)}$

Assume I am using z-score normalization. All the negative values will go to zero after the first activation layer. Isn't it true? So basically I am loosing all the information in the regions at which the signal is bellow the average.

0x90
  • 687
  • 1
  • 5
  • 17
  • Min-max scaling scales to [0, 1] (but the usual formula is that you subtract min instead of mean), so does not introduce negative values. Why negative values are a problem? – Tim Oct 16 '18 at 07:14
  • @Tim, but it's sensitive to outliers. My intuitions says that all the CNN filters will be sensitive to the sign. For example assume a standard dog/cat classification problem. Now, we will extend the inputs by adding also the $-1\cdot$ images. Will the training converge into something stable? – 0x90 Oct 16 '18 at 07:19
  • The second formula does not scale to [0,1], it would have to be $[x-\mathrm{min}(x)] / [\mathrm{max}(x) - \mathrm{min}(x)]$. Is that what you actually had in mind? – Jan Kukacka Oct 16 '18 at 07:19
  • @JanKukacka, yes. because the min could be very small. Although maybe [x−min(x)]/[max(x)−min(x)] is also a good idea. What if I create 3 channel inputs with these 3 normalization? – 0x90 Oct 16 '18 at 07:21
  • There is no requirement for neural networks that inputs need to be positive... You have weights and bias terms everywhere, so if you multiplied inputs with -1, then to get equivalent weights multiply them by -1, if you shift inputs by constant, adjust bias etc. The role of scaling is to have inputs with same magnitude. – Tim Oct 16 '18 at 07:40
  • @Tim what if you train on both the positive and the negative images. – 0x90 Oct 16 '18 at 07:43
  • I don't know what you mean by positive and negative images... You train on arrays of numbers. By very simple arithmetic operations you can easily adjust the numbers to be non-negative if needed (say by adding some constant), weights will deal with it. – Tim Oct 16 '18 at 07:47
  • @Tim assume you have a dog/cat training set. Now extend this training data by including the original dog/cat images multiplied by -1. Is it a feasible to learn such a data set using CNN. – 0x90 Oct 16 '18 at 07:55
  • Why would you do that? – Jan Kukacka Oct 16 '18 at 07:59
  • The whole idea of scaling is that all the images have same scale, so e.g. all are in [0, 1]. – Tim Oct 16 '18 at 08:03
  • @JanKukacka hypothetically. In order to see how sign effects convolutional filters – 0x90 Oct 16 '18 at 08:06

0 Answers0