This example seems to work even without “instantiate_function”.
https://diffeqflux.sciml.ai/stable/examples/neural_ode_galacticoptim/
I can’t find any docs about “instantiate_function”.
really weird, specially because instantiate_function returns itself an OptimizationFunction:
It builds the closures required for the calls to exist without the p parameters as part of it, which is required by many of the optimizers. We should probably turn that into callable structs though to help with type inference.
Thank you @ChrisRackauckas, do we have instantiate_function documentation?
No, you can make a docstring that says exactly what I said above here there. It’s an internal so it shouldn’t be in the documentation.
Sorry to bother the community with the doc questions. I think that Julia is a great technology!!! But docs is one of my biggest stoppers to use Julia in production.
When I open the official example here:
https://diffeqflux.sciml.ai/stable/examples/neural_ode_galacticoptim/
I see the the next line:
optfunc = GalacticOptim.instantiate_function(optf, prob_neuralode.p, adtype, nothing)
I guess I need to follow the official guide and use “instantiate_function” - it’s a reason why I’m asking about docs.
Oh yes, I totally forgot about that. Yes, it’s used for the AD choices these days instead of the direct OptimizationFunction definition. OptimizationFunction · GalacticOptim.jl needs to get updated, and we should turn the sections into docstrings on the dispatches. Could you help with that?
That one dispatch is just internal for wrapping the closures though and should be omitted from the docs, just get a quick comment if it’s sufficiently non-obvious.
We also need to fill out SciMLBase.jl/scimlfunctions.jl at master · SciML/SciMLBase.jl · GitHub, and I pinged @Vaibhavdixit02 for that one.
@ChrisRackauckas
I’m not sure I have a full understanding, but I created the pull request to fix that doc.
https://github.com/SciML/GalacticOptim.jl/pull/259/commits/73dee4d1348f671260362b604c9a1a9cc4b02703
Oh I see the issue. Yeah that was an old interface break.
# use GalacticOptim.jl to solve the problem
adtype = GalacticOptim.AutoZygote()
optf = GalacticOptim.OptimizationFunction((x, p) -> loss_neuralode(x), adtype)
optprob = GalacticOptim.OptimizationProblem(optf, prob_neuralode.p)
result_neuralode = GalacticOptim.solve(optprob,
ADAM(0.05),
callback = callback,
maxiters = 300)
optprob2 = remake(optprob,u0 = result_neuralode.u)
result_neuralode2 = GalacticOptim.solve(optprob2,
LBFGS(),
callback = callback,
allow_f_increases = false)
is the right code. GalacticOptim.instantiate_function shouldn’t be called by the user, so that tutorial was just doing weird stuff. I updated the tutorial and so now it should be more clearly only using the documented interface.
1 Like
I fixed the DfifEqFlux.jl documentation on that, and redid the OptimizationFunction documentation to make this more clear:
2 Likes