ForwardDiff.jl with Symbolics.jl

General Usage
, ,
1

I’m trying to do mixed symbolic and numeric derivatives. I want to define a type that holds a function that I will use to differentiate on in the future, this function may be ill-defined, so I would like to be able to operate on symbols.

In this MWE, I define an example type that stores the function. ForwardDiff works fine with the symbolic arguments as long as the function definition is exact.

using Symbolics, ForwardDiff

struct ExampleType
    func
end

function foo(obj, args)
    tempfunc(θ) = obj.func(θ...)
    return ForwardDiff.gradient(tempfunc, args)
end

myobj1 = ExampleType(+)
@variables r, θ
foo(myobj1, [r,θ])                # This works

The problem appears if I define a function that returns some sort of symbol.

function ω(r, θ)
    @variables ω(..)
    return ω(r,θ)
end
myobj2 = ExampleType(ω)
foo(myobj2, [r,θ])                # This does not work

I get the following error message when I attempt to do this.

ERROR: DimensionMismatch: gradient(f, x) expects that f(x) is a real number. Perhaps you meant jacobian(f, x)?

I’ve tried defining a custom diff_rule with no luck.

ForwardDiff.DiffRules.@define_diffrule Main.ω(r, θ) = :(Differential($r)(ω($r,$θ))), :(Differential($θ)(ω($r,$θ)))
f

Is there someway to define custom derivatives on these. sort of objects?

2

What about manually defining the dual numbers?

3

I bumped into you at juliacon and decided to try resurrecting this question. Do you mean defining a custom dual number, and overloading gradient’s behaviour on that dual number?

One thing that I tried that did work was overloading the function to act appropriately on each argument signature. Something like this worked:

function ω(r, θ)
    @variables ω(..)
    return ω(r,θ)
end
function ω(r::ForwardDiff.Dual{T}, θ) where T
    @variables ω(..)
    return ForwardDiff.Dual{T}(ω(r.value,θ), Differential(r.value)(ω(r.value,θ))*r.partials)
end
function ω(r, θ::ForwardDiff.Dual{T}) where T
    @variables ω(..)
    return Dual{T}(ω(r,θ.value), Differential(θ.value)(ω(r,θ.value))*θ.partials)
end
function ω(r::ForwardDiff.Dual{T}, θ::ForwardDiff.Dual{T}) where T
    @variables ω(..)
    return ForwardDiff.Dual{T}(ω(r.value,θ.value), Differential(r.value)(ω(r.value,θ.value))*r.partials+Differential(θ.value)(ω(r.value,θ.value))*θ.partials)
end
myobj2 = ExampleType(ω)
foo(myobj2, [r,θ])     # This worked

This solution seems to require a lot of boiler plate code though.

It also has the added issue that it doesn’t work past taking the first derivative. I seem to need to define dynamic dispatch on argument signatures associated with higher order derivatives.

4

No overloads or new types. Just define a dual number with Num values in the primal and dual parts and then stick it into the function and interpret the output. Find me at the hackathon if you need help.