In order to test the concept, I have created a toy example. I have created training data where each example is a sparse arrays of 0s and 1s. In each array, 1s appear at some regular interval. For example one array might have a 1 at every 90th place, while another array might have a 1 at every 100th place. My goal is to build a ML tool (preferably a recurrent neural network) that can detect (or classify) these evenly spaced 1s. In all the arrays, I have also added noise, meaning that I have added 1s at random places in the arrays. The ML tool should ignore these randomly placed 1s.
To demonstrate the concept, if the input is [1,1,1,0,1], then the desired output is [1,0,1,0,1]. And if the input is [0,0,1,0,1,1,1,0,1], the desired output is [0,0,1,0,0,1,0,0,1].
The above examples are simply meant to convey the concept and are not representative of my training data. The actual arrays in my training data are of length 1000, and the intervals were randomly chosen from a normal distribution centered at 100 and with a standard deviation of 33. Regarding the randomly placed 1s, the probability of a 1 appearing in any non-interval spot was 1/100.
Can a recurrent neural network (with LSTMs) do this? If not, what can? I've tried a recurrent neural network with little success. I used 2500 arrays for training data, and tested using 500 arrays. So far, the RNN hasn't worked, because it keeps predicting 0s all the time, or it just outputs an exact copy of the input. My code was:
from tensorflow.keras.layers import CuDNNLSTM
model = Sequential()
model.add(CuDNNLSTM(10, return_sequences=True))
recurrent_dropout=0.5, input_shape=(500,1)
model.add(CuDNNLSTM(10))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, batch_size=16, epochs=10, verbose=2, validation_data=(x_test, y_test), shuffle=True)
