PreallocationTools.jl with nested ForwardDiff and sparsity pattern detection errors

Specific Domains Numerics
1

Hello!

I’m trying to combine sparsity pattern detection with a nested ForwardDiff call together with PreallocationTools. A contrived, minimal example is below:

using ForwardDiff
using PreallocationTools
using Symbolics

function foo(x, cache)
    d = get_tmp(cache, x)

    d[:] = x

    0.5 * x'*x
end

function residual(r, x, cache)
    function foo_wrap(x)
        foo(x, cache)
    end
    
    r[:] = ForwardDiff.gradient(foo_wrap, x)
end

cache = DiffCache(zeros(2))

pattern = Symbolics.jacobian_sparsity((r, x) -> residual(r, x, cache), zeros(2), zeros(2))

which fails with the following stacktrace:

ERROR: ArgumentError: cannot reinterpret `Float64` as `ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#7"{DiffCache{Vector{Float64}, Vector{Float64}}}, Num}, Num, 2}`, type `ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#7"{DiffCache{Vector{Float64}, Vector{Float64}}}, Num}, Num, 2}` is not a bits type
Stacktrace:
  [1] (::Base.var"#throwbits#323")(S::Type, T::Type, U::Type)
    @ Base ./reinterpretarray.jl:16
  [2] reinterpret(#unused#::Type{ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#7"{DiffCache{Vector{Float64}, Vector{Float64}}}, Num}, Num, 2}}, a::SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true})
    @ Base ./reinterpretarray.jl:62
  [3] get_tmp(dc::DiffCache{Vector{Float64}, Vector{Float64}}, u::Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#7"{DiffCache{Vector{Float64}, Vector{Float64}}}, Num}, Num, 2}})
    @ PreallocationTools ~/.julia/packages/PreallocationTools/mJSsc/src/PreallocationTools.jl:124

Based on the discussion here I thought that I could just use cache = FixedSizeDiffCache(zeros(Symbolics.Num, 2), 2) instead in order to make non-bitstypes work, but this yields a similar error:

ERROR: ArgumentError: cannot reinterpret `ForwardDiff.Dual{nothing, Num, 2}` as `ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#13"{FixedSizeDiffCache{Vector{Num}, Vector{ForwardDiff.Dual{nothing, Num, 2}}}}, Num}, Num, 2}`, type `ForwardDiff.Dual{ForwardDiff.Tag{var"#foo_wrap#13"{FixedSizeDiffCache{Vector{Num}, Vector{ForwardDiff.Dual{nothing, Num, 2}}}}, Num}, Num, 2}` is not a bits type

I’m unsure whether this is a supported use case or not. Would anyone happen to have any insights?

2

This was a bug that is now fixed. Update packages and you should be fine.

3

Thank you for your reply! I tested the example just now with 0.4.13 and with master. I tried both DiffCache and FixedSizeDiffCache but I get the same errors in all cases.

Since this seems to be a bug, shall I open an issue on GitHub?

4

OrdinaryDiffEq v6.65?

5

Oh wait, that was a different bug with the same error message that I just happened to read on the same day. Ugh.

6

Nesting is now handled in the v0.4.15 release by More extensive handling of dispatching on type by ChrisRackauckas · Pull Request #90 · SciML/PreallocationTools.jl · GitHub

7

Cheers Chris, thank you very much! I can confirm that the minimal test case in the first post indeed works with v0.4.15 (and also v0.4.16)!

I’ll just document here in case anyone else happens to run into this issue:

Julia 1.10 was just released, and the Project.toml of both PreallocationTools and Symbolics where already updated to require the newest version. My distribution doesn’t ship 1.10 yet so I’m still on Julia 1.9. In this case, to install the newest version of the above two packages one can clone the repositories and add the local copies with Pkg. The updated Symbolics (v5.15.1) is needed as some of the relevant methods were migrated there from PreallocationTools.