0

Does using PSO have advantages/disadvantages over back-propagation when training neural networks? Please give your opinion if you have used PSO or other heuristic methods.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
siby
  • 439
  • 1
  • 4
  • 13
  • it works when the surface is rough (many local optima) or when there are discontinuities and the gradient doesn't exist. It doesn't require a traversal of the space, but it does take some work to get the "dials" of initial velocities, assuring sufficient diversity in initial positions to allow decent space traversal, particle masses, and "staying in bounds" to work well. The cloud/swarm acts like a parallel version of cross-validation, except that they interact. – EngrStudent Aug 08 '16 at 20:35
  • Thankyou all for reply. @EngrStudent you mentioned sufficient diversity in initial positions. Currently I am just initializing using a uniform distribution between 0 to 1. But the results are not very encouraging. Any input on how to properly initialize weights when using PSO. – siby Aug 08 '16 at 20:44
  • @siby - these dials interact. If the initial velocity is too high, then you can shoot your swarm "off the map". If the initial momentum is too high then the same happens. How many "individuals" in your swarm? You can look at initial position and second iteration position to see what the accelerations look like and to see if you are shooting any swarm elements out of the domain, or if they are all locked at initial position. The momentum-velocity-acceleration mix can be thought of as a numeric analog to ballistic coefficient, if that helps. – EngrStudent Aug 08 '16 at 23:09
  • 1
    Possible duplicate of [Is it possible to train a neural network without backpropagation?](https://stats.stackexchange.com/questions/235862/is-it-possible-to-train-a-neural-network-without-backpropagation) – Sycorax Jul 18 '18 at 23:44

0 Answers0