1

I have a non-convex function of several parameters, that takes real values. I am looking for a good algorithm to find local minima. I am reasonably confident I can come up with descent starting points. I have some simple boundary conditions (parameters need to be positive)

I cannot have an analytical form of the Jacobian.

I looked into Nelder-Mead, but it is said to perform poorly in high dimensions (I have about 7 parameters for my function - do not know if that qualifies as high dimension), and also I could not find an implementation in python / scipy which has constraints.

Can anyone with optimization experience suggest some good optimizing algorithms to try?

DevShark
  • 171
  • 8
  • Dimension of 7 is definitely not high. There is Nelder-Mead in [scipy](http://docs.scipy.org/doc/scipy/reference/optimize.minimize-neldermead.html) – Artem Sobolev Mar 31 '16 at 17:11
  • Ok, makes sense. Nelder-Meald still has the drawback that it does not allow expressing constraints. – DevShark Mar 31 '16 at 17:25
  • @DevShark Not if you transform the space... – Sycorax Mar 31 '16 at 17:29
  • That makes sense, I'll try taking using an exponential factor, that's a good idea. – DevShark Mar 31 '16 at 17:30
  • I've read the other question on baysian optimization (thanks for pointing it out). Do you feel it's worth using in my case? Or should I stay with Nelder-Meald? – DevShark Mar 31 '16 at 17:31
  • Nelder-Mead will be prone to getting stuck in local optima unless your choice of starting values is prescient. If you're Ok with that, no one's going to stop you. But BO is explicitly designed to solve nonconvex optimization problems, so I think it will be much more practical! – Sycorax Mar 31 '16 at 17:41
  • Ok, I'll look into BO then. I'll see if I can find an implementation in python. – DevShark Mar 31 '16 at 17:42
  • I'm not a Python user, but I think spearmint is a BO implementation. Can't vouch for its utility. – Sycorax Mar 31 '16 at 17:46
  • Try the [NLopt](http://ab-initio.mit.edu/wiki/index.php/NLopt) suite. It has Python ports. It is an excellent piece of SW. It has constrained optimization routines (eg. DIRECT, BOBYQA, etc.) applying constrains is trivial. – usεr11852 Mar 31 '16 at 19:22
  • NLopt looks great. However, it does not contain any Baysian Optimization algorithm. It contains some deterministic algos with constraints, so that's nice. – DevShark Mar 31 '16 at 21:26

0 Answers0