2

I have 2D measurement results which seem to have a constant non-trivial background as shown here:

Example for the 2D data set

For every column in this image, there seems to be a similar oscillation. I'd like to distinguish this oscillation from the real measurement signal, e.g. the oblique band which goes from about (15,1080) to (25,1140) in the image coordinates.

I already tried several things, including: FFT filtering for each individual column; using a column where no additional signal seems to be for normalization; and independent component analysis (ICA) implemented in python's scikit learn. With ICA, I was able to find a quite satisfying result, as demonstrated in this graph:

Result using ICA

where each row of the above image is plotted as a red line and the ICA result as a blue line. Unfortunately, the ICA result needs to be rescaled and it gives differing reults from time to time.

Could anyone give some hints on how to solve a task like this and how to substract the background properly and automatically in the end? Python code is highly appreciated.

Thanks!

physicsPyUser
  • 21
  • 1
  • Is this real-valued data? If you feel good about the overall shape of your background estimate (the blue line in your plot), one option would be to try calculating the [vector rejection](https://en.wikipedia.org/wiki/Vector_projection#Vector_rejection_2) of the background from each row. That is, calculate the projection of each row vector onto the background vector, then subtract that component from the row. – Jason R May 09 '16 at 13:46
  • This sounds quite good. Yes, the data is indeed real-valued. I'll give it a try, but it still doesn't solve the problem with the ICA giving me sometimes a result a, and sometime 1/a. I'd like to have an automatic approach. Thank you, anyway! – physicsPyUser May 09 '16 at 13:51
  • If your data is zero-mean, then one way to estimate the background would be to do a framewise average of your entire dataset (i.e. average down the columns of your raster image). – Jason R May 09 '16 at 13:53
  • A fully automatic background removal, what a dream! Can you share the data somehow? – Laurent Duval May 09 '16 at 17:47
  • I provided the data on a [bitbucket repository](https://cbbucket@bitbucket.org/cbbucket/data4stackexchange.git). There is only one pickle file which you can import using python pandas (`pandas.read_pickle('example_data.pkl')`). First experiments using the vector rejection look really promising. Unfortunately, I cannot provide another image due to a lack in reputation :/ – physicsPyUser May 10 '16 at 06:36

0 Answers0