Zygote Warning within FluxOptTools - 'cannot track gradients'

Specific Domains Machine Learning
,
1

I am using NNs created using Flux to parameterize a function which I want to approximate in an optimal control problem. I formulate the associated optimization problem and call Optim, which works thanks to FLuXOptTools. However, I am getting the following message:

Warning: `ForwardDiff.derivative(f, x)` within Zygote cannot track gradients with respect to `f`,   
│ and `f` appears to be a closure, or a struct with fields (according to `issingletontype(typeof(f))`).
│ typeof(f) = var"#7#8"{Vector{Float64}}
└ @ Zygote C:\Users\Gabriel\.julia\packages\Zygote\TSj5C\src\lib\forward.jl:158

Does anyone know why I am getting this? I’ve tried perusing around Zygote but couldn’t understand where this is coming from. I am afraid this has to do with second derivatives, so it ends up not affecting my algorithm since I am using first-order methods only (?).

As a matter of fact, the objective function decreases as well as the gradient norm, but it gets stuck halfway through. Most likely a numerical issue, and not related to the warning above, but figured I would share nonetheless… Thank you!

2

This is in dire need of a MWE. It’s most likely that some code is inside the function being optimized is using ForwardDiff without your knowledge. This may be intentional (on the library author’s part) or due to a misunderstanding of the higher-level libraries.

3

It was indeed a ForwardDiff I had in my code. Not in the library, but in my own code. It was conflicting with the backward pass from Zygote. Thank you for clarifying.