4

Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day.

I'm looking for good linear and nonlinear optimization textbooks for self learning. Here are some of the criteria (subjective) for a good optimization book:

  1. Written in a didactic style or problem based learning approach with examples. if the algorithm is iterative in nature, should at least have first 2 iteration in the form of hand calculation. Theorems and poofs are great, but have having numerical examples improves learning
  2. Present optimization algorithms in Pseudo code format not as a paragraph or in words.
  3. Computer programs should accompany text in high level language such as Matlab, Mathematica or Python.
  4. Present traditional unconstrained optimization methods such as gradient descent, quasi newton, conjugate gradient, and constrained optimization such as quadratic programming, sequential quadratic programming and reduced gradient methods. Should cover both line search and trust region methods.
  5. Also present nature inspired methods such as evolutionary computing, and other techniques for nonconvex optimization since most real world problems one cannot expect convexity and often noisy, discrete and/or discontinuous.
  6. Should be written using English language and letters as opposed to Greek letters.

Any recommendations?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
forecaster
  • 7,349
  • 9
  • 43
  • 81
  • 2
    You might want to check this thread on [References on numerical optimization for statisticians](https://stats.stackexchange.com/questions/8570/references-on-numerical-optimization-for-statisticians). That said, I do not think a book fulfilling all the requirement you list exists. Maybe K. Lange's or J. Gentle's books might be close to what you want but they do not have accompanying code nor cover evolutionary computing and that stuff... Hard ask, I am curious for any solid answers. – usεr11852 Oct 13 '18 at 23:55
  • What is wrong with Greek letters? Avoiding them in mathy texts seems quite a tough requirement. – Richard Hardy May 31 '19 at 06:30
  • Did you find any textbooks you liked? I am looking for something similar, too. – Richard Hardy Oct 22 '19 at 10:40
  • 1
    @RichardHardy yes, [Algorithms for Optimization (The MIT Press)](https://www.amazon.com/dp/0262039427/ref=cm_sw_em_r_mt_dp_U_SjVRDb75DRXYK). Outstanding book written by engineers, all psedocodes are written in Julia programming language. Julia has the ability to work with Greek letters so the psuedocodes has been directly translated to the Julia programs. – forecaster Oct 22 '19 at 12:23
  • Thank you for your quick response! This is among the books I am considering, too. Why not post it as an answer to your own question? – Richard Hardy Oct 22 '19 at 12:27

0 Answers0