# DifferentiationInterface not generating correct shadow

**URL:** <https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868>\
**Category:** Optimization (Mathematical)\
**Tags:** question, optimization, enzyme\
**Created:** [August 26, 2025, 7:42pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868 "2025-08-26T19:42:11Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexl123](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@alexl123](https://discourse.julialang.org/u/alexl123)\
**Post date:** [August 26, 2025, 7:42pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/1 "2025-08-26T19:42:11Z")

</div>

Hello!  
I am trying to play with the Sophia optimization algorthim. I am storing my parameters as a ComponentArray since some parameters are scalar and others are vectors. When I use a first order method (Adam) I have no issue, but when I use a SecondOrder adtype I get an error. Below is a MWE that maintains the structure of my real code. The addition u\_buffer is for hacking together TBPTT.

```julia-auto
using Enzyme
using SciMLSensitivity
using Optimisers
using DiffEqFlux
using ComponentArrays
using OrdinaryDiffEq
using Optimization
using DifferentiationInterface

function test_dynamics!(du, u, p, t)
    a = p.a; b = p.b
    du[1] = a[1] * u[1] + b * u[2]
    du[2] = a[2] * u[2] + b * u[1]
end
function dist_loss!(p, u_buffer, u_initial, prob,dt)
    x0 = 1f0; y0 = 2f0
    sol = solve(prob, Tsit5(),u0=u_initial, p=p, saveat=dt, sensealg = GaussAdjoint(autojacvec=EnzymeVJP()))#, sensealg=SensitivityADPassThrough())
    dists=zeros(Float32, length(sol.u))
    for k in 1:length(sol.u)
        dist_sq = (sol.u[k][1]-x0)^2 + (sol.u[k][2]-y0)^2
        dists[k] = dist_sq
    end
    total_loss = sum(dists) * Float32(1e-6)
    u_buffer .= sol.u[end]
    return Float32(total_loss)
end

params = ComponentArray(a=[1f0, 2f0], b=3f0)
u0 = [1f0, 1f0]; u0_buff = similar(u0)
prob = ODEProblem(test_dynamics!, u0, (0f0, 1f0), params)
loss_fn = (p, hp) -> dist_loss!(p, u0_buff, u0, prob, 0.1f0)

adtype = Optimization.AutoEnzyme(; mode=Enzyme.set_runtime_activity(Enzyme.Reverse))
second_ad = DifferentiationInterface.SecondOrder(adtype, adtype) # inner & outer

optf = Optimization.OptimizationFunction(loss_fn, second_ad)
optprob = Optimization.OptimizationProblem(optf, params)

result = Optimization.solve( optprob, Optimization.Sophia(), maxiters = 10)

```

and this is the error

> MethodError: no method matching Duplicated(::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, ::Vector{Float32})  
> The type `Duplicated` exists, but no method is defined for this combination of argument types when trying to construct it.  
> Closest candidates are:  
> Duplicated(::T1, !Matched::T1) where T1  
> @ EnzymeCore ~/.julia/packages/EnzymeCore/lmG5F/src/EnzymeCore.jl:68  
> Duplicated(::T1, !Matched::T1, !Matched::Bool) where T1  
> @ EnzymeCore ~/.julia/packages/EnzymeCore/lmG5F/src/EnzymeCore.jl:68  
> Duplicated(!Matched::AbstractLuxLayer, !Matched::AbstractLuxLayer)  
> @ LuxCoreEnzymeCoreExt ~/.julia/packages/LuxCore/SN4dl/ext/LuxCoreEnzymeCoreExt.jl:23  
> Stacktrace:  
> [1] value\_and\_pullback!(::typeof(DifferentiationInterface.shuffled\_gradient!), ::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, ::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, ::DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient!), ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}, ::AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, ::Tuple{Vector{Float32}}, ::DifferentiationInterface.FunctionContext{var"#981#982"}, ::DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, ::DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, ::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterfaceEnzymeExt ~/.julia/packages/DifferentiationInterface/zJHX8/ext/DifferentiationInterfaceEnzymeExt/reverse\_twoarg.jl:140  
> [2] pullback!(::typeof(DifferentiationInterface.shuffled\_gradient!), ::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, ::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, ::DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient!), ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}, ::AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, ::Tuple{Vector{Float32}}, ::DifferentiationInterface.FunctionContext{var"#981#982"}, ::DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, ::DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, ::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/zJHX8/src/first\_order/pullback.jl:557  
> [3] \_hvp\_aux!(::DifferentiationInterface.InPlaceSupported, f::var"#981#982", tg::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, prep::DifferentiationInterface.ReverseOverReverseHVPPrep{Tuple{var"#981#982", SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseOneArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient), AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient!), ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}}, backend::SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, x::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, tx::Tuple{Vector{Float32}}, contexts::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/zJHX8/src/second\_order/hvp.jl:733  
> [4] hvp!(f::var"#981#982", tg::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, prep::DifferentiationInterface.ReverseOverReverseHVPPrep{Tuple{var"#981#982", SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseOneArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient), AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Tuple{typeof(DifferentiationInterface.shuffled\_gradient!), ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, Tuple{Vector{Float32}}, Tuple{DifferentiationInterface.FunctionContext{var"#981#982"}, DifferentiationInterface.Constant{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, DifferentiationInterface.Constant{DifferentiationInterface.Rewrap{1, Tuple{typeof(DifferentiationInterface.constant\_maker)}}}, DifferentiationInterface.Constant{SciMLBase.NullParameters}}}, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}}, backend::SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, x::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, tx::Tuple{Vector{Float32}}, contexts::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/zJHX8/src/second\_order/hvp.jl:713  
> [5] hvp!(f::var"#981#982", tg::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, backend::SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, x::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, tx::Tuple{Vector{Float32}}, contexts::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/zJHX8/src/second\_order/hvp.jl:89  
> [6] (::OptimizationBase.var"#hv!#24"{SciMLBase.NullParameters, DifferentiationInterface.ReverseOverReverseHVPPrep{Nothing, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseOneArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}}, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}})(H::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, θ::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, v::Vector{Float32}, p::SciMLBase.NullParameters)  
> @ OptimizationBase ~/.julia/packages/OptimizationBase/1tTb9/src/OptimizationDIExt.jl:105  
> [7] \_\_solve(cache::OptimizationCache{OptimizationFunction{true, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, var"#981#982", OptimizationBase.var"#grad#16"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, OptimizationBase.var"#fg!#18"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, OptimizationBase.var"#hess#20"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}}, Nothing, OptimizationBase.var"#hv!#24"{SciMLBase.NullParameters, DifferentiationInterface.ReverseOverReverseHVPPrep{Nothing, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseOneArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}}, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, Optimization.Sophia, Bool, var"#973#974", Nothing})  
> @ Optimization ~/.julia/packages/Optimization/hfEs4/src/sophia.jl:108  
> [8] solve!(cache::OptimizationCache{OptimizationFunction{true, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, var"#981#982", OptimizationBase.var"#grad#16"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, OptimizationBase.var"#fg!#18"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, OptimizationBase.var"#hess#20"{SciMLBase.NullParameters, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}}, Nothing, OptimizationBase.var"#hv!#24"{SciMLBase.NullParameters, DifferentiationInterface.ReverseOverReverseHVPPrep{Nothing, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseOneArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, DifferentiationInterfaceEnzymeExt.EnzymeReverseTwoArgPullbackPrep{Nothing, Nothing, NTuple{4, Nothing}, Tuple{Vector{Float32}}}}, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, Optimization.Sophia, Bool, var"#973#974", Nothing})  
> @ SciMLBase ~/.julia/packages/SciMLBase/rvXrA/src/solve.jl:226  
> [9] solve(::OptimizationProblem{true, OptimizationFunction{true, SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, var"#981#982", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT\_OBSERVED\_NO\_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, @Kwargs{}}, ::Optimization.Sophia; kwargs::@Kwargs{maxiters::Int64, callback::var"#973#974"})  
> @ SciMLBase ~/.julia/packages/SciMLBase/rvXrA/src/solve.jl:128

I would’ve thought that the higher level functions would’ve created a ComponentArray for the shadow.

Am I doing something wrong or is this a bug?

Thanks for your time and effort

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [August 26, 2025, 8:29pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/2 "2025-08-26T20:29:10Z")

</div>

@ChrisRackauckas this seems like a bug in OptimizationBase.jl, Optimization shouldn’t use DI for Enzyme?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 26, 2025, 10:13pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/3 "2025-08-26T22:13:23Z")

</div>

> [@alexl123](#):
>
> [5] hvp!(f::var"#981#982", tg::Tuple{ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}}, backend::SecondOrder{AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}, AutoEnzyme{ReverseMode{false, true, false, FFIABI, false, false}, Nothing}}, x::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{(a = ViewAxis(1:2, Shaped1DAxis((2,))), b = 3)}}}, tx::Tuple{Vector{Float32}}, contexts::DifferentiationInterface.Constant{SciMLBase.NullParameters})  
> @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/zJHX8/src/second\_order/hvp.jl:89

Judging by this frame of the stack trace, `DI.hvp!` is called with `x::ComponentVector` but `tx::Tuple{Vector}`, which suggests that somewhere in the Optimization stack the type of the input is not respected?

> [@wsmoses](#):
>
> @ChrisRackauckas this seems like a bug in [OptimizationBase.jl](https://juliaregistries.github.io/General/packages/redirect_to_repo/OptimizationBase), Optimization shouldn’t use DI for Enzyme?

If I understand correctly, OptimizationBase does have Enzyme-specific implementations for `backend::AutoEnzyme`, but here the user supplied `backend::DI.SecondOrder`, which is why those dispatches were not touched.

As a side remark, @alexl123 do you have any reason to choose reverse-over-reverse for your second-order backend? You’d probably be better off with

```julia
forward_adtype = Optimization.AutoEnzyme(; mode=Enzyme.set_runtime_activity(Enzyme.Forward))
reverse_adtype = Optimization.AutoEnzyme(; mode=Enzyme.set_runtime_activity(Enzyme.Reverse))
second_ad = DifferentiationInterface.SecondOrder(forward_adtype, reverse_adtype) # inner & outer

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 26, 2025, 10:18pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/4 "2025-08-26T22:18:35Z")

</div>

After taking a closer look at Sophia, maybe the problem is here?

> <https://github.com/SciML/Optimization.jl/blob/8f8e4ffb2cb59b6f78fc93b26a532febecd76f9f/src/sophia.jl#L154-L155>

It seems to me that this generates a `Vector` for `u` regardless of the type of `θ` (in this case `ComponentVector`). Other autodiff backends may not raise an eyebrow but Enzyme’s `Duplicated` system enforces type consistency between primal and shadow, so that’s where you see the error but that’s not where it originates.

A simple way to preserve type information would be to use `randn!` and initialize `u` properly at the beginning:

```julia
julia> using ComponentArrays, Random

julia> x = ComponentVector(a=[1.0, 2.0])
ComponentVector{Float64}(a = [1.0, 2.0])

julia> randn!(x)
ComponentVector{Float64}(a = [0.07972696817241108, -0.935076728780384])

```

---

<div class="post-metadata">

**Author:** ![alexl123](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@alexl123](https://discourse.julialang.org/u/alexl123)\
**Post date:** [August 26, 2025, 10:40pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/5 "2025-08-26T22:40:30Z")

</div>

Thank you!

I am still very new to AD, so no particular reason for reverse over reverse. I just want to use Sophia, I only explicitly created the SecondOrder AD type because I got a warning suggesting I do that.

Should I modify sophia.jl locally to implement that change?

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [August 26, 2025, 11:38pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/6 "2025-08-26T23:38:40Z")

</div>

Ah I see, equivalently here I would recommend just passing in

```julia-auto
adtype = Optimization.AutoEnzyme(; mode=Enzyme.set_runtime_activity(Enzyme.Reverse))

```

directly instead of using the DI.secondorder (which will force it to use the official native Enzyme implementation directly).

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 27, 2025, 8:09am UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/7 "2025-08-27T08:09:52Z")

</div>

At the moment this will yield the same behavior for second-order optimization algorithms inside Optimization.jl.

```julia
julia> result = Optimization.solve(optprob, Optimization.Sophia(); maxiters=10)
┌ Warning: The selected optimization algorithm requires second order derivatives, but `SecondOrder` ADtype was not provided. 
│ So a `SecondOrder` with AutoEnzyme(mode=ReverseMode{false, true, false, FFIABI, false, false}()) for both inner and outer will be created, this can be suboptimal and not work in some cases so 
│ an explicit `SecondOrder` ADtype is recommended.
└ @ OptimizationBase ~/.julia/packages/OptimizationBase/Lc8sB/src/cache.jl:49
ERROR: MethodError: no method matching Duplicated(::ComponentVector{Float32, Vector{Float32}, Tuple{Axis{…}}}, ::Vector{Float32})
The type `Duplicated` exists, but no method is defined for this combination of argument types when trying to construct it.

```

See also:

- [Choice of second-order AD and warnings · Issue #970 · SciML/Optimization.jl · GitHub](https://github.com/SciML/Optimization.jl/issues/970)
- [Warning secondorder ADtype in optimization.jl - #3 by JADekker](https://discourse.julialang.org/t/warning-secondorder-adtype-in-optimization-jl/122539/3)

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [August 27, 2025, 11:45am UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/8 "2025-08-27T11:45:13Z")

</div>

Oh hm, that definitely seems like an Optimization.jl bug then, passing in an incorrect type

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 27, 2025, 3:59pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/9 "2025-08-27T15:59:50Z")

</div>

> [@alexl123](#):
>
> Should I modify sophia.jl locally to implement that change?

Even better, you could try to contribute a fix!  
While you’re at it, it would also be nice to be able to pass an RNG to Sophia for reproducibility

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 1, 2025, 4:20pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/10 "2025-09-01T16:20:53Z")

</div>

Should be handled on latest release now.

---

<div class="post-metadata">

**Author:** ![alexl123](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@alexl123](https://discourse.julialang.org/u/alexl123)\
**Post date:** [September 3, 2025, 8:35pm UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/11 "2025-09-03T20:35:55Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![johtok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johtok/32/218643_2.png) [@johtok](https://discourse.julialang.org/u/johtok)\
**Post date:** [January 4, 2026, 9:01am UTC](https://discourse.julialang.org/t/differentiationinterface-not-generating-correct-shadow/131868/12 "2026-01-04T09:01:42Z")

</div>

Worked for me on 1.11.8 😃