1

I'm having the following scenario:

I'm working with Deep Learning for solving seismic processing tasks. The seismic data may be understood as a collection of images that represents a given sub-surface region. Kind of an x-ray but to see the underground. To simplify, it is a discrete cube $C$, and each sample $s \in C$ are real numbers.

A seismic cube can be seen as a 3D matrix of real values:

enter image description here

My question is in regard to the best practices for normalizing such kind of dataset for training and inferencing.

Usually, the neural network (NN) receives batches of sections (images) of this volumes. But it is unfeasible to load this volume beforehand to compute their min/max values to perform some kind of normalization and standardization. And we know that ideally, the NN should receive samples (images) such that their values are in the range $[0,1]$.

What is the recommended practice to train the NN in this situation?

And after I got the trained network, how to treat the new images for inference?

João Paulo Navarro
  • 183
  • 1
  • 4

1 Answers1

2

Normalization/standardization isn't usually performed batch-wise or image-wise, but on the whole dataset.

This means that any measure you need to perform the scaling (min & max for normalization, mean & std for standardization) should be measured on the whole training set and not for each image or batch separately. After obtaining these measures, you perform the scaling on each image independently.

When performing inference, each image is normalized/standardized with the measures calculated from the training set.

I don't see how that makes any difference in your case. You should normalize/standardize each input separately from the measures calculated from all images on the training set.

Djib2011
  • 5,395
  • 5
  • 25
  • 36
  • 1
    Djib2011 thanks for your reply. I got the idea of normalization using the training dataset values. But in practice, this is not feasible for me, because I'm going to have terabytes of data, and not all will be used for training. Do you believe that applying image/batch normalization could deliver sub-optimal results in such a critical way? – João Paulo Navarro Sep 13 '18 at 15:01
  • 2
    @JoãoPauloNavarro you could try calculating the global measures by opening the data in batches and for example for the global mean averaging the means of the batches. I have tried a more local normalization (each batch separately) but the results I got were worse than even without any normalization. – Djib2011 Sep 13 '18 at 17:15