# ForwardDiff tagcount with custom diff rules

**URL:** <https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534>\
**Category:** Numerics\
**Tags:** differentiation, forwarddiff\
**Created:** [July 26, 2024, 11:05pm UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534 "2024-07-26T23:05:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dchang10](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dchang10/32/44162_2.png) [@dchang10](https://discourse.julialang.org/u/dchang10)\
**Post date:** [July 26, 2024, 11:05pm UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534/1 "2024-07-26T23:05:13Z")

</div>

I’m writing a custom `gradient` for a function that takes two derivatives using `ForwardDiff`. I receive an error unless I specify the tag for the returned `Dual`. I figured that the tag I should return depends on the tag count, but it’s not clear to me whether I should attach the tag with the larger or the smaller count. Here’s an implementation that seems to work, though I’m not sure if it’s correct.

```julia
function JacobiElliptic.CarlsonAlg.E(x::ForwardDiff.Dual{T1}, y::ForwardDiff.Dual{T2}) where {T1, T2}
    xval = x.value
    yval = y.value

    ∂yE = iszero(yval) ? -π/8 : (JacobiElliptic.CarlsonAlg.E(xval, yval)-JacobiElliptic.CarlsonAlg.F(xval, yval))/(2yval)
    return ForwardDiff.Dual{ForwardDiff.tagcount(T1) < ForwardDiff.tagcount(T2) ? T1 : T2}(JacobiElliptic.CarlsonAlg.E(xval, yval), ForwardDiff.Partials(((sqrt(1-yval*sin(xval)^2)), ∂yE)))
end

```

Which is the correct tag that I should attach to this method?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 27, 2024, 8:44am UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534/2 "2024-07-27T08:44:24Z")

</div>

Hi! Can you share a complete example with the error you get from ForwardDiff?

---

<div class="post-metadata">

**Author:** ![dchang10](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dchang10/32/44162_2.png) [@dchang10](https://discourse.julialang.org/u/dchang10)\
**Post date:** [July 27, 2024, 2:45pm UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534/3 "2024-07-27T14:45:23Z")

</div>

I get the following error if I return a `Dual` without specifying the tag:

```julia
function JacobiElliptic.CarlsonAlg.E(x::ForwardDiff.Dual{T1}, y::ForwardDiff.Dual{T2}) where {T1, T2}
    xval = x.value
    yval = y.value

    ∂yE = iszero(yval) ? -π/8 : (JacobiElliptic.CarlsonAlg.E(xval, yval)-JacobiElliptic.CarlsonAlg.F(xval, yval))/(2yval)
    ForwardDiff.Dual(JacobiElliptic.CarlsonAlg.E(xval, yval), ForwardDiff.Partials(((sqrt(1-yval*sin(xval)^2)), ∂yE)))
end

```

```julia
ERROR: Cannot determine ordering of Dual tags Nothing and ForwardDiff.Tag{typeof(func), Float64}
Stacktrace:
 [1] ≺(a::Type, b::Type)
   @ ForwardDiff ~/.julia/packages/ForwardDiff/PcZ48/src/dual.jl:54
 [2] partials
   @ ~/.julia/packages/ForwardDiff/PcZ48/src/dual.jl:108 [inlined]
 [3] extract_gradient!(::Type{ForwardDiff.Tag{…}}, result::Vector{Float64}, dual::Dual{Nothing, Float64, 2})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/PcZ48/src/gradient.jl:64
 [4] vector_mode_gradient(f::typeof(func), x::Vector{…}, cfg::ForwardDiff.GradientConfig{…})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/PcZ48/src/gradient.jl:92
 [5] gradient(f::Function, x::Vector{…}, cfg::ForwardDiff.GradientConfig{…}, ::Val{…})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/PcZ48/src/gradient.jl:0
 [6] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{…}, Float64, 2, Vector{…}})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/PcZ48/src/gradient.jl:17

```

My main confusion is that I’m not sure which tag to return since I’ve defined the function to act on two separate `Dual`s.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [July 27, 2024, 2:56pm UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534/4 "2024-07-27T14:56:02Z")

</div>

I think you probably want this:

```julia
function JacobiElliptic.CarlsonAlg.E(x::ForwardDiff.Dual{T}, y::ForwardDiff.Dual{T}) where {T}
    xval = x.value
    yval = y.value

    ∂yE = iszero(yval) ? -π/8 : (JacobiElliptic.CarlsonAlg.E(xval, yval)-JacobiElliptic.CarlsonAlg.F(xval, yval))/(2yval)
    ForwardDiff.Dual{T}(JacobiElliptic.CarlsonAlg.E(xval, yval), ForwardDiff.Partials(((sqrt(1-yval*sin(xval)^2)), ∂yE)))
end

```

Two different tags means these are independent perturbations, which will occur when taking second derivatives – and will result in nested Dual types:

```julia
julia> using ForwardDiff: Dual, Tag, derivative

julia> Dual(1,2) + Dual(3,5) # first derivative, both arguments perturbed together
Dual{Nothing}(4,7)

julia> Dual{Tag{1,1}}(1,2) + Dual{Tag{2,2}}(3,5) # second derivative, two indep perturbations
Dual{Tag{2, 2}}(Dual{Tag{1, 1}}(4,2),Dual{Tag{1, 1}}(5,0))

julia> Dual{Tag{2,2}}(1,2) + Dual{Tag{1,1}}(3,5) # same, but organised differently
Dual{Tag{2, 2}}(Dual{Tag{1, 1}}(4,5),Dual{Tag{1, 1}}(2,0))

```

---

<div class="post-metadata">

**Author:** ![dchang10](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dchang10/32/44162_2.png) [@dchang10](https://discourse.julialang.org/u/dchang10)\
**Post date:** [July 27, 2024, 4:00pm UTC](https://discourse.julialang.org/t/forwarddiff-tagcount-with-custom-diff-rules/117534/5 "2024-07-27T16:00:26Z")

</div>

Thanks for that