Note that I am not currently using StaticArrays (but instead MVector and co), so from what I can see I should not hit those methods.
I have tried RFLUFactorization, but like other methods using LU I have not found a way to pass the LinearSolve.init cache to DiffererentiationInterface.jacobian!. In particular even if I manually (though in the most naive way) define recursive_similar via type piracy for the involved structs (very much going against the warning from the docs that JClugstor mentioned), I still get warnings of the form
MethodError: no method matching lu!(::MMatrix{3, 3, ForwardDiff.Dual{…}, 9}, ::Vector{ForwardDiff.Dual{…}}, ::Val{true}, ::Val{true}; check::Bool)
The DualLinearCache (and in general the machinery defined in LinearSolveForwardDiffExt.jl ) you both mentioned seems interesting. Is this something I might want to wrangle manually (using something like Base.get_extension(LinearSolve, :LinearSolveForwardDiffExt).DualLinearCache)? I assume all of this would require me to abandon using DifferentiationInterface anyway.