Composing Lux with Optim Algorithms

Is there a way to use optimization algorithms from Optim to train Lux neural networks and neural ODEs? In particular, I would like to use LBFGS to train a Lux neural network but don’t see a simple way to do this. There is a library called FluxOptimTools that allows us to do this in Flux.

See Neural Ordinary Differential Equations · DiffEqFlux.jl. You can essentially use the Optimization.jl API to train Lux using Optim (and several other optimization packages out there)

I see, but there is no way to write a general training loop from which we could call Optimisers.update with an algorithm like LBFGS.

Optim.jl is completely distinct from Optimisers.jl (maintained by different groups of people). Optim.jl doesn’t provide a way to apply updates like Optimisers sequentially. But in my experience, you can always use callbacks to do most things you need from a custom training loop.