# ForwardDiff & "complicated" functions?

**URL:** <https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094>\
**Category:** General Usage\
**Created:** [November 14, 2019, 5:59pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094 "2019-11-14T17:59:02Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 14, 2019, 5:59pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/1 "2019-11-14T17:59:03Z")

</div>

I’d need to compute the derivative (and later: the Jacobian) of the following function:

```julia
f = x -> sqrt(max(-x,0))

```

which looks like the following:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/1/013a156006091bd34b6e16f4ede188467ee63bc5.png)  
Obviously, the derivative doesn’t exist for x=0. Still, I try to use `ForwardDiff`:

```julia
fd = x -> ForwardDiff.derivative(f,x)

```

I then try to plot this function, leading to:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/3/83cc280fbe8bb88a4821781f8d48886803d3df43.png)  
Upon checking, it turns out that fd(x) gives value `NaN` for x\in[0,\infty).  
I then try to define the function as follows:

```julia
g = x -> begin
    if x < 0
        sqrt(abs(x))
    else
        0
    end
end

```

and define the derivative as

```julia
gd = x -> ForwardDiff(g,x)

```

OK – this derivative doesn’t evaluate…

```julia
julia> gd(-2.)
MethodError: objects of type Module are not callable

Stacktrace:
 [1] (::getfield(Main, Symbol("##42#43")))(::Float64) at .\In[113]:1
 [2] top-level scope at In[130]:1

```

Questions:

1. Why doesn’t evaluation of `gd(-2.)` work?
2. Why does `fd(x)` evaluate to `NaN` for x\in [0,\infty)?
3. Is there any other package that can handle this problem?
4. …or is it better to insert a smooth transition from, say f(-10^{-6}), to f(0) or f(10^6)?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 14, 2019, 6:17pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/2 "2019-11-14T18:17:59Z")

</div>

You are not calling the package functions right. See the docs:

[http://www.juliadiff.org/ForwardDiff.jl/latest/user/api.html#ForwardDiff.derivative](http://www.juliadiff.org/ForwardDiff.jl/latest/user/api.html#ForwardDiff.derivative)

Also, you may want to make `g` type stable. Eg

```julia
using ForwardDiff
g(x) = x < 0 ? sqrt(abs(x)) : zero(x)
ForwardDiff.derivative(g, -1)
ForwardDiff.derivative(g, 1)

```

---

<div class="post-metadata">

**Author:** ![ssfrr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ssfrr/32/3736_2.png) [@ssfrr](https://discourse.julialang.org/u/ssfrr)\
**Post date:** [November 14, 2019, 6:22pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/3 "2019-11-14T18:22:19Z")

</div>

> [@BLI](#):
>
> gd = x → ForwardDiff(g,x)

This should be `ForwardDiff.derivative(g,x)`, (probably a typo?) which is causing your “Module not callable” error.

You can use the `@which` macro to see where the function `max` is defined which might explain the `NaN`.

As a side note - the style `f = x-> ...` is not very idiomatic in Julia. Usually that would be written as `f(x) = ...`. For multi-line functions folks generally use the style:

```julia
function f(x)
    ...
end

```

---

<div class="post-metadata">

**Author:** ![ssfrr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ssfrr/32/3736_2.png) [@ssfrr](https://discourse.julialang.org/u/ssfrr)\
**Post date:** [November 14, 2019, 6:31pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/4 "2019-11-14T18:31:33Z")

</div>

actually the `@edit` macro might be more useful. Running `@edit max(1.0, 0)` brings you to the promote code that turns `0` into a float, so `@edit max(1.0, 1.0)` gives the code from `math.jl`:

```julia
max(x::T, y::T) where {T<:AbstractFloat} = ifelse((y > x) | (signbit(y) < signbit(x)),
                                    ifelse(isnan(x), x, y), ifelse(isnan(y), y, x))

```

Something in there (maybe the `isnan`?) is making `ForwardDiff` unhappy.

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 14, 2019, 9:00pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/5 "2019-11-14T21:00:38Z")

</div>

Ah. A typo.

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 15, 2019, 12:59pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/6 "2019-11-15T12:59:10Z")

</div>

I’m trying to linearize an ODE model…`using ForwardDiff`, but don’t get the expected result. A simple example:

```julia
# Model with input functions
function watertank(dT,T,p,t,u)
    # Parameters
    g,h,d,per,A,rho,chp,Ux,Po = p
    # Inputs
    uP,uv,Ti,Ta,VdL = u
    # 
    md = rho*uv*VdL
    # 
    As = 2*A + per*h
    V = A*h
    m = rho*V
    Cp = m*chp
    Qd_el = Po*uP
    Qd_a = Ux*As*(Ta-T[1])
    dT[1] = (md*chp*(Ti-T[1]) + Qd_el + Qd_a)/Cp
    #
    return dT
end

```

Data for the model:

```julia
# -- constants
g = 9.81
h = 1.5
d = 0.5
per = pi*d
A = pi*d^2/4
rho = 1.0e3
chp = 4.19e3
Ux = 0.45
Po = 15e3
#
p = [g,h,d,per,A,rho,chp,Ux,Po];
#
T_op = [25.]
u_op = [0.5,0.5,28.,20.,8e-3/60];

```

Here, `T_op` and `u_op` constitute the operating point for the linearization. Next, I define functions which only depend on `T` and `u`:

```julia
wt_T(T) = watertank(copy(T),T,p,0,u_op)
wt_u(u) = watertank(copy(T_op),T_op,p,0,u)

```

These work as expected:

```julia
julia> wt_T(T_op)
1-element Array{Float64,1}:
 0.006751564884010684

```

and similarly for `wt_u(u_op)`.  
Next, I introduce Jacobians:

```julia
J_T(T) = ForwardDiff.jacobian(wt_T,T)
J_u(u) = ForwardDiff.jacobian(wt_u,u)

```

The Jacobian wrt. `T` works as expected:

```julia
julia> J_T(T_op)
1×1 Array{Float64,2}:
 -0.00022735608347665157

```

HOWEVER, the Jacobian wrt. `u` gives an error mesage:

```julia
julia> J_u(u_op)
MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5})
Closest candidates are:
  Float64(::Real, !Matched::RoundingMode) where T<:AbstractFloat at rounding.jl:194
  Float64(::T<:Number) where T<:Number at boot.jl:718
  Float64(!Matched::Int8) at float.jl:60
  ...

Stacktrace:
 [1] convert(::Type{Float64}, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5}) at .\number.jl:7
 [2] setindex!(::Array{Float64,1}, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5}, ::Int64) at .\array.jl:766
 [3] watertank(::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::Int64, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5},1}) at .\In[2]:16
 [4] wt_u(::Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5},1}) at .\In[4]:3
 [5] vector_mode_jacobian(::typeof(wt_u), ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5},1}}) at C:\Users\Bernt_Lie\.julia\packages\ForwardDiff\N0wMF\src\apiutils.jl:37
 [6] jacobian(::Function, ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5},1}}, ::Val{true}) at C:\Users\Bernt_Lie\.julia\packages\ForwardDiff\N0wMF\src\jacobian.jl:17
 [7] jacobian(::Function, ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(wt_u),Float64},Float64,5},1}}) at C:\Users\Bernt_Lie\.julia\packages\ForwardDiff\N0wMF\src\jacobian.jl:15 (repeats 2 times)
 [8] J_u(::Array{Float64,1}) at .\In[4]:4
 [9] top-level scope at In[8]:1

```

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 15, 2019, 1:15pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/7 "2019-11-15T13:15:08Z")

</div>

I think that for `wt_u`, `dT` is an array of `Float64`s, so it cannot accommodate the `ForwardDiff.Dual`s.

You could make your code non-modifying (ie reallocate a new `dT` of the right type), or just calculate `dT[1]` as a scalar variable and return that (as far as I understand, that’s the only thing that is changed).

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 15, 2019, 1:42pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/8 "2019-11-15T13:42:57Z")

</div>

Hm. “Jacobian” normally involves a vector valued function in several variables. Doesn’t `dT` being an array simply mean that the function is vector valued?

The documentation says that the function should return an `AbstractArray`… [This method assumes that `isa(f(x), AbstractArray)`]. Is `Float64` a subset of `AbstractArray`?

---

<div class="post-metadata">

**Author:** ![DrPapa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drpapa/32/6835_2.png) [@DrPapa](https://discourse.julialang.org/u/DrPapa)\
**Post date:** [November 15, 2019, 1:59pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/9 "2019-11-15T13:59:39Z")

</div>

@BLI this doesn’t directly address your issue with ForwardDiff, but you could consider using ModelingToolkit.jl to specify the ODE and get the Jacobian that way.

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 15, 2019, 2:25pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/10 "2019-11-15T14:25:36Z")

</div>

I could probably use ModelingToolkit.jl. I’m really interested in ModelingToolkit.jl, but It is not really documented for “the average user” at the moment.

I can, of course, find the Jacobian _numerically_:

```julia
B = zeros(length(T_op),length(u_op))
Iu = Diagonal(ones(length(u_op)))
for j in 1:length(u_op)
    B[:,j] = (wt_u(u_op+Iu[:,j]*1e-6) - wt_u(u_op))/1e-6
end

```

but it would be much more elegant to use ForwardDiff or some other tool.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 15, 2019, 3:13pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/11 "2019-11-15T15:13:40Z")

</div>

> [@BLI](#):
>
> I could probably use ModelingToolkit.jl. I’m really interested in ModelingToolkit.jl, but It is not really documented for “the average user” at the moment.

It’s part of the tutorial now:

[http://docs.juliadiffeq.org/latest/tutorials/advanced\_ode\_example/#Automatic-Derivation-of-Jacobian-Functions-1](http://docs.juliadiffeq.org/latest/tutorials/advanced_ode_example/#Automatic-Derivation-of-Jacobian-Functions-1)

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 15, 2019, 3:42pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/12 "2019-11-15T15:42:12Z")

</div>

I’ll look into it. But I need something more general… I’m considering systems with inputs, \frac{dx}{dt} = f(x,u;t) and I also need the Jacobian wrt. the input, i.e., I need both \frac{\partial f}{\partial x} and \frac {\partial f}{\partial u}. The DiffEq solvers assume that I have made the mapping g(x,t) = f(x,u(x,t),t), so I assume that ModelingToolkit will find the Jacobian of g(x,t) wrt. x, only.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 15, 2019, 3:50pm UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/13 "2019-11-15T15:50:08Z")

</div>

Oh yes, if your parameters are like that you can use ModelingToolkit to derive the Jacobian, but the automatic version needs some work to handle your case.

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [November 16, 2019, 5:52am UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/14 "2019-11-16T05:52:35Z")

</div>

this works without modification of `watertank`

```julia
wt_T!(dT,T) = watertank(dT,T,p,0.0,u_op)
wt_u!(dT,u) = watertank(dT,T_op,p,0.0,u)
J_T(dT,T) = ForwardDiff.jacobian(wt_T!,dT,T)
J_u(dT,u) = ForwardDiff.jacobian(wt_u!,dT,u)
J_u(u) = J_u(copy(T_op),u)
J_T(T) = J_u(copy(T),T)

```

the main problem was treating a function ´f!(result,x)´ as ´f(x)´, but ForwardDiff can work with mutable functions if you specify that:

```julia
julia> J_u(u_op)
1×5 Array{Float64,2}:
 0.012155 0.00135812 0.000226354 1.00239e-6 5.09296

```

now, i want to see that `cp` as a function of `T` 😄

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [November 16, 2019, 10:15am UTC](https://discourse.julialang.org/t/forwarddiff-complicated-functions/31094/15 "2019-11-16T10:15:42Z")

</div>

Thanks! I’ll check it out!