An unexpected clash between NonlinearSolve and ForwardDiff

ij05 · May 8, 2026, 9:26am

Hello,

I was successfully using the following block of code which uses ForwardDiff.gradient on a function, that has a Nonlinearsolve.NonlinearLeastSquaresProblem. It was giving out correct gradients (and jacobians) until recently, and now, it no longer works. I would really appreciate any help in figuring out why this is happening.

Here is a MWE

using ForwardDiff, LinearAlgebras
import NonlinearSolve as NLS

l1 = 10.0;
l2 = 5.0;

function objective!(F,u,p)
   th1 = u[1];
   th2 = u[2];

    x = p[1]; y = p[2];

    F[1] = l1*cos(th1)+l2*cos(th1+th2)-x;
    F[2] = l1*sin(th1)+l2*sin(th1+th2)-y;

    return F
end

function cog_pos(pos)

    T = eltype(pos)

    u0 = T[0.0; 0.0]; # initial guess
    prob2 = NLS.NonlinearLeastSquaresProblem(objective!, u0, pos)
    sol = NLS.solve(prob2)    
    
    th1 = sol.u[1];
    th2 = sol.u[2];    

    cog_pos_x = 0.5*l1*cos(th1);
    

    return cog_pos_x
end

p = [5.0; 7.0];
ForwardDiff.jacobian(cog_pos,p)

I get a very large MethodError: no method matching Float64, which I am not sharing here. The stacktrace is as follows:

Stacktrace:
  [1] convert(::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBaseForwardDiffExt.var"#6#7"{NonlinearSolveBase.var"#nlls_generate_vjp_function##8#nlls_generate_vjp_function##9"{typeof(objective!), SciMLBase.NonlinearSolution{Float64, 1, Vector{Float64}, Vector{Float64}, SciMLBase.NonlinearLeastSquaresProblem{Vector{Float64}, true, Vector{Float64}, SciMLBase.NonlinearFunction{true, SciMLBase.AutoSpecialize, NonlinearSolveBase.AutoSpecializeCallable{FunctionWrappersWrappers.FunctionWrappersWrapper{Tuple{FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, Vector{Float64}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{Float64}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}}}}, FunctionWrappersWrappers.AllowNonIsBits, FunctionWrappersWrappers.SingleCacheStorage}}, UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing}, @Kwargs{}, Nothing, Nothing}, NonlinearSolveBase.NonlinearSolvePolyAlgorithm{Val{6}, Tuple{NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{Missing, Missing, NonlinearSolveBase.NewtonDescent{Nothing}, Nothing, Nothing, Nothing, Nothing, Val{false}}, NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{Missing, NonlinearSolveFirstOrder.LevenbergMarquardtTrustRegion{Float64}, NonlinearSolveBase.DampedNewtonDescent{Nothing, Float64, NonlinearSolveFirstOrder.LevenbergMarquardtDampingFunction{Float64, Float64, Float64}}, Nothing, Nothing, Nothing, Nothing, Val{true}}, NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{Missing, NonlinearSolveFirstOrder.GenericTrustRegionScheme{NonlinearSolveFirstOrder.RadiusUpdateSchemes.__Simple, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}}, NonlinearSolveBase.Dogleg{NonlinearSolveBase.NewtonDescent{Nothing}, NonlinearSolveBase.SteepestDescent{Nothing}}, Nothing, Nothing, Nothing, Nothing, Val{false}}, NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{LineSearch.BackTracking{Nothing, Float64, Float64, Float64, Val{3}, Float64, Bool}, Missing, NonlinearSolveBase.NewtonDescent{Nothing}, Nothing, Nothing, Nothing, Nothing, Val{false}}, NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{Missing, NonlinearSolveFirstOrder.GenericTrustRegionScheme{NonlinearSolveFirstOrder.RadiusUpdateSchemes.__Bastin, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}, Rational{Int64}}, NonlinearSolveBase.Dogleg{NonlinearSolveBase.NewtonDescent{Nothing}, NonlinearSolveBase.SteepestDescent{Nothing}}, Nothing, Nothing, Nothing, Nothing, Val{false}}, NonlinearSolveFirstOrder.GeneralizedFirstOrderAlgorithm{Missing, NonlinearSolveFirstOrder.LevenbergMarquardtTrustRegion{Float64}, NonlinearSolveBase.GeodesicAcceleration{NonlinearSolveBase.DampedNewtonDescent{Nothing, Float64, NonlinearSolveFirstOrder.LevenbergMarquardtDampingFunction{Float64, Float64, Float64}}, Float64, Float64}, Nothing, Nothing, Nothing, Nothing, Val{true}}}}, Any, Nothing, SciMLBase.NLStats, NonlinearSolveBase.NonlinearSolveTrace{Val{false}, Val{false}, Nothing, NonlinearSolveBase.NonlinearSolveTracing{Val{:minimal}}, SciMLBase.NonlinearLeastSquaresProblem{Vector{Float64}, true, Vector{Float64}, SciMLBase.NonlinearFunction{true, SciMLBase.AutoSpecialize, NonlinearSolveBase.AutoSpecializeCallable{FunctionWrappersWrappers.FunctionWrappersWrapper{Tuple{FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, Vector{Float64}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{Float64}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}, Vector{Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolveBase.NonlinearSolveTag, Float64}, Float64, 1}}}}}, FunctionWrappersWrappers.AllowNonIsBits, FunctionWrappersWrappers.SingleCacheStorage}}, UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing}, @Kwargs{}, Nothing, Nothing}}}, ADTypes.AutoForwardDiff{nothing, Nothing}, Nothing}, Vector{Float64}}, Float64}, Float64, 2})
    @ Base .\number.jl:7
  [2] setindex!
    @ .\array.jl:985 [inlined]
  [3] macro expansion
    @ .\broadcast.jl:995 [inlined]
  [4] macro expansion
    @ .\simdloop.jl:77 [inlined]
  [5] copyto!
    @ .\broadcast.jl:994 [inlined]
  [6] copyto!
...
    @ ForwardDiff C:\Users\user\.julia\packages\ForwardDiff\z3jRk\src\jacobian.jl:19
 [40] jacobian(f::typeof(cog_pos), x::Vector{Float64})
    @ ForwardDiff C:\Users\user\.julia\packages\ForwardDiff\z3jRk\src\jacobian.jl:19
 [41] top-level scope
    @ c:\Users\user\Documents\final chapter julia report\jl_notebook_cell_df34fa98e69747e1a8f8a730347b8e2f_X21sZmlsZQ==.jl:1
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

The output of Pkg.status() is

  [26cc04aa] FiniteDifferences v0.12.33
  [f6369f11] ForwardDiff v1.3.3
  [e9467ef8] GLMakie v0.13.10
  [ee78f7c6] Makie v0.24.10
  [8913a72c] NonlinearSolve v4.19.0
  [1ed8b502] SciMLSensitivity v7.106.0
  [e88e6eb3] Zygote v0.7.10
  [37e2e46d] LinearAlgebra v1.12.0

ChrisRackauckas · May 8, 2026, 10:40am

Yes, I’m trying to figure out if it’s possible to work around, or if we need to do a bugfix in ForwardDiff

github.com/JuliaDiff/ForwardDiff.jl

Precompilation can lead to Dual tag ordering problems

opened 03:03PM - 29 Oct 24 UTC

Technici4n

Hi, I found an issue when working with precompilation and nested derivatives. … The tag count system is setup such that the inner derivative receives a tag with a higher `tagcount` than the outer derivative. When precompiling, it can happen that only the outer derivative is precompiled, for example because of runtime dispatch that depends on the value of some parameters. When running the code later, the atomic counter for tags starts from 0 again, which can cause the inner derivative to be given a tag with a lower count. This leads to a `Cannot determine ordering of Dual tags` error. Here is a reproduction example: https://github.com/Technici4n/FDPrecompilationIssue. Precompilation should succeed, and running the test should fail with the following output: ``` Testing Running tests... Tag count of Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,1) is 1 Tag count of Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#1#2"{ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}}(Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,0),Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,2)) is 0 ERROR: LoadError: Cannot determine ordering of Dual tags ForwardDiff.Tag{FDPrecompilationIssue.var"#1#2"{ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}} and ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64} ``` (I encountered this issue was encountered in the context of https://github.com/mfherbst/AiidaDFTK.jl but the setup is more complicated than in this minimal reproduction example. AiidaDFTK precompiles a stress computation (outer derivative) using the LDA functional (does not require a nested derivative). I encountered the issue when running a stress computation through AiidaDFTK using a PBE functional (which requires a nested gradient). You can think of `α` from the reproduction example as the parameter that selects the functional, LDA as `α = 0`, PBE as `α = 1`.)

github.com/JuliaDiff/ForwardDiff.jl

Fix #714: Make TAGCOUNT always greater than the last instantiated Tag's count (#724)

master ← Technici4n:fix-714

opened 09:16AM - 18 Nov 24 UTC

Technici4n

+3 -1

This fixes #714: ``` Testing Running tests... Tag count of Dual{ForwardD…iff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,1) is 1 Tag count of Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#1#2"{ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}}(Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,0),Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,2)) is 2 Testing FDPrecompilationIssue tests passed ``` This adds an atomic operation for each `Tag` instantiation. This might be problematic when computing many fast derivatives in a tight loop?

It’s a rather tricky one, but I’ve had some solutions open for a few days and I just need to pick one by tomorrow.

Topic		Replies	Views
ForwardDiff.jl Jacobian with NonlinearSolve.jl General Usage forwarddiff , nonlinearsolve	1	93	September 13, 2025
Method Error for ForwardDiff.hessian when ForwardDiff.Jacobian and ForwardDiff.gradient works perfectly fine. How to resolve? New to Julia question , forwarddiff	18	479	June 7, 2024
How to solve a nonlinear problem Numerics question , package , nonlinearsolve	5	260	March 13, 2025
Issues with computing gradient with ForwardDiff.jl (Any fixes other than ND?) Machine Learning forwarddiff , neural-network	2	239	December 24, 2023
My function works on it's own but not in NonLinearSolve General Usage forwarddiff , nonlinearsolve	10	613	April 22, 2024

An unexpected clash between NonlinearSolve and ForwardDiff

Related topics