[ANN] Optim.jl updates

Package Announcements
21

Links

Release notes:
v1.13.2, v1.13.3, v2.0.0, v2.0.1, v2.1.0, v2.2.0, v2.2.1

Registry PRs:
v1.13.2, v1.13.3, v2.0.0, v2.0.1, v2.1.0, v2.2.0, v2.2.1

Updated docs: Home · Optim

Description

It has been a while since the last update here, so this post catches up on everything since v1.13.1. There is quite a lot, including the v2.0.0 breaking release, so I have split it into the releases that came before v2.0.0 and the v2.x line.

Pre-v2.0

v1.13.2 and v1.13.3 were both small in terms of number of changes but important fixes to clarity and correctness. v1.13.2 fixed some typos in the BFGS docs (thanks @cgarling, #1175). v1.13.3 NLSolversBase API fixes and cleanup after a LineSearches release was made to be more robust agains shadowing of values that could get out of sync and preparing for directional derivative support in LineSearches (@devmotion, #1213).

v2.0.0 and the v2.x line

v2.0.0 was of course more significant. The main user facing changes were:

  • Automatic differentiation now only goes through the ADTypes and DifferentiationInterface ecosystem rather than the old symbol based interface (#1195). If you were passing autodiff as a symbol, have a look at the updated docs.
  • Callbacks now always run and receive the optimization state, and that functionality is documented. An important decoupling of callbacks and trace printing (#1184, #1224, #1226).
  • Jacobian vector products are now supported in the line searches. This makes algorithms that needs to update the directional derivative (MoreThuente, HagerZhang, StrongWolfe,…) more efficient if you can write your directional derivative as a more optimized calculation than g’d (#1210).
  • You can now build an options set from an existing one with Options(options; kws...) (#1186, #1172).
  • NewtonTrustRegion was improvements and the default minimum trust region radius is now 0 (#1215, #1219).
  • A fair amount of cleanup: we stopped overloading Base.minimum and Base.maximum, depend on Statistics instead of StatsBase, and added Aqua, ExplicitImports and JET tests. Big thanks to @devmotion for carrying most of this work.

v2.0.1 was a quick follow up. It re-instates the old Fminbox convergence behavior and fixes a value_gradient! issue for ManifoldObjective (#1232, fixes #1228 and #1229).

v2.1.0 brought various fixes to the MOI wrapper (@odow, #1235), exposed beta in the state of the NLCG solver (@nmbader, #1238), added a restart of the optimization when the line search fails (#1242), moved to LineSearches 6.7.2 (#1253), and included several documentation updates (@abhro).

v2.2.0 Was a significant feature release with a full L-BFGS-B implementation in Optim (#1258) that I have been walking to and from for years. Also there were updates to the conjugate gradient method to use the theta parameter from Hager & Zhang (2013) along with a betamax to bound the beta value (@nmbader, #1259).

v2.2.1 is a targeted bugfix from @RafaelArutjunjan (#1257). It guards against gbarrier_norm == 0 when computing the initial mu, which resolves #1233 where the default Fminbox.mu0 = NaN could cause optimization failures.

All the best,
Patrick

22

Follow-up: obvious giant omission to tag and thank @gdalle who’s helped throughout with anything related to DifferentiationInterface.jl