Questions regarding a few ForwardDiff.jl limitations

General Usage
1

I have a project I’m wanting to use ForwardDiff.jl on. I have a few question:

Chunk Size

  • If I’m reading correctly, ForwardDiff.jl evaluates partial derivatives on chunks of the input vector. What if computations depend on the whole input vector? A trivial example:
function f(x)
  output = 0.0
  if sum(x) > 5.0
    for i = 1:length(x)
      output += sin(x[i])
    end
  else
    for i = 1:length(x)
      output += cos(x[i])
    end
  end
end
  • According to the docs, “each differentiation of a chunk requires […] additional memory proportional to the square of the chunk’s size.” Does this mean if it’s chunked into 10 64-bit elements, the memory allocated is (10*10)^2*64 bits or (10*64)^2 bits?

External C-Calls: I understand why these aren’t permitted for calculations, but are they permitted for non-calculation purposes (e.g. producing index ordering for a permuted vector)?

Types: What exactly has to be typed to accept Real? Just everything the input vector “touches” or absolutely everything within the function?

Skipping Operations: I have an iterative solver within my objective function. I see no reason why I’d need to track all differentiations within this. Is there a way to skip this section of the objective function?

2

If no dual number does a ccall you’re fine. It sounds like your examples will be fine.

Everything that gets touched by a Dual number. It needs to be able to, well, accept a Dual number.

Just do a type-check for Dual numbers, but you should be sure that it won’t effect the derivative.

3

Regarding chunking, the entire input vector is always passed into the function, regardless of the chunk size. The chunk size is just the number of partial derivatives which are evaluated during each call to the function. So the size of the input will be the number of elements in x times the chunk size.