Benchmarking Optimization Heuristics

Does anyone have benchmarks/success stories in running Heuristics in Julia vice other Languages (Python, R, etc.)? I’m on a new team (so I don’t want to make too many bold changes) and a firm believer in Julia, looking for any documented support to argue a move to Julia.

Thanks!

Julia packages exist for most well-known algorithms, but they are not magically better than the same algorithms in other languages. If evaluating the objective is costly, Julia may be faster, but that’s not related to optimization.

I would suggest that you invest in experimenting with what’s available in Julia before recommending that your team adopt the language. Success stories of others will help little without having someone on the team who is familiar with the language and the relevant packages.

I am not sure what kind of support you are expecting here; most packages are maintained by researchers who found some algorithms useful. If you need the kind of paid support that goes beyond what can be expected in free software, you may have to just approach various package maintainers individually.

You might have a look at, Case Study - Julia Computing

I don’t have a specific write up but I can say anicotically that a significant chunk of my job is developing handcrafted heuristics for specific types of optimization problems, both for publication and deployment. One thing I found really wonderful about Julia is that I can start developing high-level algorithmic ideas with minimal engineering overheads (e.g. ignore types, memory allocation, ect…). Once the algorithm design is solidified I used Julia’s benchmarking tools I can identify the core bottlenecks and improve the performance as-needed with typing and other code optimization tricks. In my experience, very carefylly written Julia can be comparable to C++ in performance.

This work flow is very different than my past experience when I would first develop in python (or similar) and then have to start over with C++ once we knew what algorithm needed to be built. Sticking in one language and only focusing on the core bottlenecks saves me lots of time.

I am also a big fan of the JuMP modeling layer for optimization. I very often use JuMP models as a sub-problem in more complex heuristics and the JuMP overheads are almost never the bottleneck in my workflows.

2 Likes