0

I'm a beginner in machine learning und I have a problem to find the best way to compare the performance (accuracy) of different SVMs and CNNs (Jupyter Notebook)? The CNNs I train in Google Colab with GPU.

So far I've tried to make the models reproducible by setting seeds und train each model architecture several times. The reproducibility works with the SVMs, but not with the CNNs in Google Colab. But generally I'm not sure if reproducibility is really helpful in this case.

Wouldn't it be better not to put any seeds and train each model several times and then compare the models by using the calculated mean and standard deviation?

I hope and would be very grateful when you can explain me how to do it right.

Many thanks in advance

mdewey
  • 16,541
  • 22
  • 30
  • 57
Code Now
  • 37
  • 4

1 Answers1

0

If CNN returns random results, there must be the part related to RNG. If it's hard to locate where exactly they are, you can just try several times and compare them with mean and STD.

However, I wouldn't recommend you to do that b/c it might take a long time to train CNN. Rather, I recommend you to read the documents of ML module that you are using (i.e. Tensorflow, PyTorch, Keras, ...) to find out where the RNG is coming from.

crux26
  • 101
  • 5
  • I use Keras with Tensorflow backend and Jupyter notebook for the CNNs. My problem seems to be already a big topic of discussion. So far there seems to be no solution for my ML modules. – Code Now Oct 20 '19 at 11:54
  • The only clue I keep coming across is to use the CPU instead of the GPU. But this solution would really take a lot of time for the CNNs. – Code Now Oct 20 '19 at 11:59