Debugging non-finite Jacobian elements (w/ ForwardDiff)

Specific Domains Numerics
,
1

I have a rather complex calculation that maps a vector of floats to a longer vector of floats.

At some inputs, ForwardDiff gives me a Jacobian where some elements are NaN. Inputs are of course finite, and so is the function value.

When checked with FiniteDifferences and that works fine. I would like to understand what is going on (I am suspecting a numerical issue with a rule or something). I have thought of adding

_finiteD(x::Real) = true
_finiteD(x::ForwardDiff.Dual) = all(isfinite, x.partials)
_finiteD(a::AbstractArray) = all(_finiteD, a)

and then peppering the code with

@assert _finiteD(some_interim_value)

but this will be tedious. Is there a shortcut?

2

maybe use DifferentiationInterface to try different backends?

1 Like
3

Sometimes using NaN safe mode has fixed such issues for me, Advanced Usage Guide · ForwardDiff.

But otherwise I’ve done as you and resorted to manually print if I detect nan/inf and then figure out why. Sometimes, it was needed to modify the equation with eps() (to avoid division by zero, for example, for square roots). Would be nice if there was an automatic detection that did require so much manual insertion of checks!

1 Like
4

I tried Enzyme which fails (reported the issue), is there any other backend suitable for forward mode?