Using ForwardDiff to calculate gradients of fields in composite type?



Hello everybody,

A little background:

I’m working on a package that implements creates differentiable McCormick relaxations of functions. These relaxations are generally tighter than interval bounds and since they are convex/concave in construction they are useful in a branch-and-bound optimization framework. The base object is for these are SMC:

struct SMC{T} <: AbstractMC{T}
  cc::T # concave relaxation value
  cv::T # convex relaxation value
  Intv::Interval{T} # interval over which the relaxation is defined

A library of operations is then constructed in a typical operator overloading fashion. So if one wanted to compute the relaxations of exp(-x^2)*sin(x) at x = 2 in the interval [1,3], you’d use the following code:

x = SMC{Float64}(2,2,Interval(1,3))
y = exp(-x^2)*sin(x)
convex =
concave =

I’d like to be able to get gradients of and via automatic differentiation (forward or mixed type). So ultimately, I’d like to setup functions like below and use a differentiation tool to calculate gradients.

using ForwardDiff
using IntervalArithmetic

function example(x::Vector)
   Xbox = IntervalBox(-1..1,-2..2)
   temp1 = SMC{Float64}(x[1],x[1],Xbox[1])
   temp2 = SMC{Float64}(x[1],x[1],Xbox[1])
   temp3 = exp(temp1*temp2)

g = x -> ForwardDiff.gradient(example, x);
x0 = [1.0,2.0]
g(x0) $ gradient

Is this type of automatic differentiation feasible with ForwardDiff in concept?

I’ve been trying to get this style working for a bit and I’ve run into a lot of conversion errors with regard to Dual Numbers. So I just want to make sure this is possible before I wade deeper into this problem.

I’m almost ready to publish this package and I’ve got a separate package which includes gradient objects in SMC and naively does forward differentiation. So I can share that shortly.

Thanks for the time and consideration.


Yes, this is definitely possible. You just have to be careful to make your code generic enough. For example, the SMC constructor should be deriving its type parameter from its inputs x[1] etc rather than being fixed to Float64, otherwise it won’t be able to accept the dual number type that ForwardDiff passes in.