NLsolve, preallocated ForwardDiff Dual

Specific Domains Optimization (Mathematical)
1

I’m trying to use NLsolve (using ForwardDiff) to solve a system of equations that produces intermediate output. For performance, I’m trying to preallocate arrays for it to store that intermediate output, but I’m not sure what type to give those arrays. I know it needs to be a ForwardDiff.Dual{T, V, N}, where V will be Float64 and N will be the number of variables. But I’m not sure what tag to use. I tried Any, but that appears to be wholly incompatible with whatever tags are assigned by the gradient calls occurring inside NLsolve. Is there a recommended way around this? I’ve been told I should turn tag checking off, but I’m not sure how to do that, when I don’t issue the gradient calls myself. Any thoughts would be quite helpful.

2

We ended up needing something similar in RigidBodyDynamics. The solution that we came up with was to create a cache, essentially a dictionary from element type to pre-allocated vectors. To get your pre-allocated vector, you just check if it’s in the dictionary, using the existing vector if it’s found or creating and storing a new one otherwise.

This seems type-unstable, but it actually works well because we can add a type-annotation on the result we get from the cache. For example:

function my_inner_loop(cache::Dict{DataType, Any}, x::Vector{T}) where T
  preallocated = get!(cache, T) do
    Vector{T}(N)
  end::Vector{T}  # note the extra type annotation here
  # You can now use `preallocated` and your code should be type-stable
  ...
end

There were a few tricks to get better performance, so you might want to check out the discussion here: https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/363#issuecomment-345804093

3

We have a section of the DiffEq FAQ which discusses this:

http://docs.juliadiffeq.org/latest/basics/faq.html#I-get-Dual-number-errors-when-I-solve-my-ODE-with-Rosenbrock-or-SDIRK-methods…?-1

4

For reference, at least today, the new link for this is I get Dual number errors when I solve my ODE with Rosenbrock or SDIRK methods