Treating Variable as Constant in ForwardDiff

Specific Domains Numerics
1

I’ve distilled my use case into a simple example. Essentially, I have an outer function that takes an input and generates a scalar from an inner function. I want ForwardDiff to treat this scalar as a constant when taking the derivative of the outer function, but ForwardDiff is (correctly) tracing the input through the inner function. How can I get ForwardDiff to “ignore” the inner function? I’ve tried deep-copying, but that doesn’t do the trick. Some example code:

using ForwardDiff

function f1(x)
    return g(x)
end

function f2(x)
    scalar = deepcopy(g(x));
    return scalar;
end

function g(x)
    return x^2
end

ForwardDiff.derivative(f1,3) # => 6
ForwardDiff.derivative(f2,3) # => 6, want this to be 0

In f2, I want ForwardDiff to great g(x) as a constant, and thus the derivative with respect to x should be zero. Thanks in advance for any help!

1 Like
2

This is kind of a strange thing to do: after all, the derivative of f2(x) w.r.t. x really is 3. While it’s certainly possible to overload f2(::ForwardDiff.Dual) and intercept just the value of x, this feels like the wrong solution.

Instead, how about just being explicit about what quantities are being differentiated? For example, you can split apart the input you want to differentiate from some other constant input using a closure:

julia> g(x) = x^2
g (generic function with 1 method)

julia> function f2(x, y)
         z = g(x)
         z + y
       end
f2 (generic function with 1 method)

# Take the derivative of f2(x, y) w.r.t. y, evaluated at y = 3, with x fixed at 3
julia> ForwardDiff.derivative(y -> f2(3, y), 3)
1
2 Likes
3

You’re right, it is odd in my example! In case you’re interested, imagine a problem in economics where an agent’s optimization should not take into account that their action affects the aggregate economy. (This isn’t what I’m doing in this project, but the idea is the same.) In that case, if x is one of the agent’s idiosyncratic state variables, you’d want to use x to calculate some aggregate quantity, A(x), but taking derivatives with respect to the state in the consumer problem, A(x) is held fix.

Your suggestion works but it makes things a little unwieldily in my project. Thanks! Hopefully there’s a “better” way to do it, but if not, this is great.