When using Optimization.jl to perform function minimization or weight evaluation in neural networks using Lux, how can I access the iteration number and other optimization characteristics that vary in time from the callback function? I am currently inserting data I need in a global dictionary. But my specific problem is accessing the iteration number in the optimization procedure. Any advice? Thanks.
It’s not accessible in the solution object right now, but could be. Open an issue for it, but that will take a bit. In the mean time, just use a Ref and count calls of callbacks or the objective function, depending on what you’re trying to measure.
Since you can access the original result from the optimizer in the original field of the OptimizationSolution if the optimizer library stores these fields that you need you can access them from there.
Example with Optim
using Optimization, ForwardDiff, Zygote
rosenbrock(x, p) = (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
x0 = zeros(2)
_p = [1.0, 100.0]
f = OptimizationFunction(rosenbrock, Optimization.AutoForwardDiff())
l1 = rosenbrock(x0, _p)
prob = OptimizationProblem(f, x0, _p)
## Optim.jl Solvers
using OptimizationOptimJL
# Start with some derivative-free optimizers
sol = solve(prob, SimulatedAnnealing())
sol.original
sol.original.iterations