I have been using Python’s scipy.optimize.minimize(method="LBFGSB") for my research, and have been looking to speed the code because it doesn’t scale. And so I tried to rewrite my code in Julia using Optim.jl, with Optim.LBFGS as the method. Unfortunately, my situation is the opposite of Optimize p…

[image] rht: . Optim.jl is ~15x slower is it due to compilation latency? Here’s my post on this topic Julia vs R vs Python: simple optimization | Codementor

I used @btime, which has conveniently excluded compilation latency.

Things to rule out: Is Optim.jl taking qualitatively different steps than your Python code? Is the minimizer you reach different than what Python produces? Is Optim pushing for higher accuracy than Python is? Are the gradients being used different? In general, focusing on performance optimization…

Is Optim.jl taking qualitatively different steps than your Python code? Optim.jl did 3 iterations, scipy.optimize did 4 iterations. Optim.jl did 3833 function calls, scipy.optimize did 186!! Optim.jl defaults to ftol = 0.0, scipy.optimize defaults to ftol = 2.220446049250313e-09. Optim.jl def…

There might be deeper issues here.

It is possible that the number of line searches per iteration is different. Apparently scipy defaults to a maximum of 20. I could not find quickly to what Optim defaults. It is not clear to me if either allows you to change that. That could explain making one more iteration and declaring convergenc…

Have you tried using scipy.optimize through PyCall. Rather than keeping both .py and .jl versions. You only keep the Julia code and use scipy when you need. In the models I have worked, scipy through Pycall is actually faster than when I use Optim.jl. Don’t know why, but that’s the way it is.

Regarding with line search, what is the algorithm used by scipy.optimize? https://github.com/scipy/scipy/blob/4ec6f29c0344ddce80845df0c2b8765fc8f27a72/scipy/optimize/lbfgsb_src/lbfgsb.f#L762-L829 contains no reference to any paper. [image] jovansam: In the models I have worked, scipy through Py…

[image] lmiq: (What kind of gradient is being used in each case?) Is this dictated by the L-BFGS-B algorithm, or are users supposed to have freedom in specifying their own gradient function?

Optim.jl vs scipy.optimize once again

General Usage Performance

nilshg May 23, 2021, 7:33am 5

Looking in the opposite direction to John, have you benchmarked how long a single evaluation of your objective function takes in Python vs Julia?

Topic		Replies	Views
Making LBFGS in `Optim.jl` as fast as in `NLopt.jl` Optimization (Mathematical)	13	2489	December 6, 2022
Optimize performance comparison - Optim.jl vs Scipy Performance	15	6043	February 12, 2020
Performance of Optim.jl vs. Matlab's fminunc Optimization (Mathematical) performance	12	4268	August 6, 2018
Is there a Julia equivalent of scipy.optimize.minimize(method='TNC')? Optimization (Mathematical)	23	2818	August 21, 2020
Optimization, from MATLAB to Julia Optimization (Mathematical) matlab , optimization	10	1168	April 13, 2024

Optim.jl vs scipy.optimize once again

Related topics