Derivative of linear function

New to Julia
1

How to balance the variants type in the simple case below:

function dev(X::Number)
return diff(X)
end

if Julia derivatives X with respect to X returns 1.0?

ERROR: LoadError: MethodError: no method matching diff(::Float64)
Closest candidates are:
diff(!Matched::AbstractRange{T}; dims) where T at multidimensional.jl:851
diff(!Matched::SparseArrays.AbstractSparseMatrixCSC; dims) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/SparseArrays/src/linalg.jl:1068
diff(!Matched::AbstractArray{T,1} where T) at multidimensional.jl:809

2

diff computes the difference between elements in a vector, e.g.

julia> x = [1, 2, 5];

julia> diff(x)
2-element Vector{Int64}:
 1
 3

This operation is not defined on a single number, thats why you get a MethodError.

2 Likes
3

FWIW, for “numerical” derivatives, you can use ForwardDiff.jl.

using ForwardDiff
import ForwardDiff.derivative
f(x) = x

julia> derivative(f, 2.0)
1.0

For symbolic derivatives, use Symbolics.jl.

4

I would say that ForwardDiff computes the exact symbolic derivative, though you can only directly access the result evaluated using floating-point arithmetic. If you look at the assembly code the compiler generates from ForwardDiff, it contains the exact symbolic derivative formula; for example, here it is computing that the derivative of x^2 is 2x:

julia> f(x) = x^2

julia> @code_llvm f(3.0)
define double @julia_f_202(double %0) {
    %1 = fmul double %0, %0
  ret double %1
}

julia> @code_llvm ForwardDiff.derivative(f, 3.0)
     %1 = fmul double %0, 2.000000e+00
  ret double %1
}

Whereas “numerical derivatives” typically refer to approximate derivative formulas obtained from a sequence of function values only, such as those computed by FiniteDifferences.jl.

4 Likes
5

Thank you for the insight. According to this Wikipedia article, automatic differentiation as used in ForwardDiff is distinct from symbolic differentiation and numerical differentiation.

Btw, why the right-limit is output for the following derivative at zero?

using ForwardDiff
import ForwardDiff.derivative
f(x) = abs(x)

julia> derivative(f, 0)
1.0

And interestingly the left-limit can be obtained too:

julia> derivative(f, -0.0)
-1.0

While strictly speaking, the derivative at 0 does not exist.

Thank you.

2 Likes
6

Most automatic differentiation systems take the point of view that they can essentially return anything at a non-differentiable point, and should prefer to return a value that is useful. There’s currently a ChainRulesCore.jl pull request to document this convention, here is a preview of the proposed documentation.

https://juliadiff.org/ChainRulesCore.jl/previews/PR419/nondiff_points.html

6 Likes