1

I have read that deep learning models outperforms than the traditional machine learning models.

I have a time-series classification problem where the output is 0 or 1. I used LSTM to classify my timeseries as follows. 

model = Sequential()
model.add(Conv1D(10, kernel_size=3, input_shape=(25,4)))
model.add(Conv1D(10, kernel_size=2))
model.add(GlobalMaxPooling1D())
model.add(Dense(10))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Unfortunately, my deep learning model gives very bad results (e.g., 0.333333). I am worried why this happens. Then I tried a machine learning model (using randomforest) and it gave accuracy about 0.6.

I am upsetting why the deep learning model gives that bad results. I would like to get your feedback on why this happens, and is there a way to avoid this.

I am happy to provide more details if needed.

EmJ
  • 592
  • 3
  • 15
  • 1
    1) It looks like you used a convolutional neural network, not LSTM. 2) Are you talking about performance on training data or on data not used to build the model? – Dave Oct 31 '19 at 07:50
  • @Dave I am using 10-fold cross validation. It would be really great if you could tell me how I can use CNN for a classification problem as I have never used CNN before. I look forward to hearing from you. Thank you :) – EmJ Oct 31 '19 at 10:21
  • 1
    1) I would be interested in the accuracy you get on training data. Do you know the term "overfitting"? 2) You just did use a CNN. If you want to use LSTM, don't use CNN. (Okay, they can be combined, but at least make sure to use LSTM if you want to have long short-term memory in your network). If you're unfamiliar with the idea behind CNN, Brandon Rohrer has an excellent video on YouTube: https://www.youtube.com/watch?v=FmpDIaiMIeA. He also has a longer one that I have not seen: https://www.youtube.com/watch?v=JB8T_zN7ZC0. – Dave Oct 31 '19 at 10:56
  • 2
    [Why is accuracy not the best measure for assessing classification models?](https://stats.stackexchange.com/q/312780/1352) – Stephan Kolassa Oct 31 '19 at 11:12

1 Answers1

5

it's not true that deep learning models outperforms the traditional machine learning or even statistical models. It all depends on the problem, what kind of data you have, how many, what's the dimensional etc.

In general, DL might outperform other models in situations where you have a lot of data (10k, 100k, 1m) and when the model is very nonlinear such as in computer vision applications. In other settings DL models are not necessarily as great.

For example in time series forecasting competition, Makridakis et al. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889 the best machine learning model including LSTM was worse than the worst statistical model.

rep_ho
  • 6,036
  • 1
  • 22
  • 44