1

I'm interested in neural networks from a general machine learning and pattern recognition perspective and not as much from the perspective of image processing or NLP data. If I want to train a neural network with many layers each of which have many neurons (maybe 50 layers of 100 neurons each), then will a GPU speed up the process of learning?

I've heard everyone say that GPUs speed up everything, but I'm wondering if there is much more benefit for convolutional layers, or if GPUs will always speed up matrix multiplication. Basically, in practice, what advantage can GPUs give a person training a non-convolutional neural network?

JacKeown
  • 628
  • 1
  • 6
  • 17
  • 4
    This is really more a question for stack overflow or the data science stack exchange since it has nothing to do with stats. However, GPUs speed up all types of neural network layers, not just convolutional ones. They speed things up by being more efficient at matrix multiplication which all types of layers use. – Barker Apr 23 '19 at 21:22
  • 2
    I think this is not an off-topic question in the sense of requiring basic programming assistance; instead, it is asking for practical considerations regarding applied machine learning and as such should be on topic here. The discussion about similar questions is [open on meta](https://stats.meta.stackexchange.com/questions/5633/are-questions-concerning-deployment-best-practices-on-topic) btw. – Jan Kukacka Apr 24 '19 at 11:40

0 Answers0