Derivative of linear function

How to balance the variants type in the simple case below:

function dev(X::Number)
return diff(X)

if Julia derivatives X with respect to X returns 1.0?

ERROR: LoadError: MethodError: no method matching diff(::Float64)
Closest candidates are:
diff(!Matched::AbstractRange{T}; dims) where T at multidimensional.jl:851
diff(!Matched::SparseArrays.AbstractSparseMatrixCSC; dims) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/SparseArrays/src/linalg.jl:1068
diff(!Matched::AbstractArray{T,1} where T) at multidimensional.jl:809

diff computes the difference between elements in a vector, e.g.

julia> x = [1, 2, 5];

julia> diff(x)
2-element Vector{Int64}:

This operation is not defined on a single number, thats why you get a MethodError.


FWIW, for ‚Äúnumerical‚ÄĚ derivatives, you can use ForwardDiff.jl.

using ForwardDiff
import ForwardDiff.derivative
f(x) = x

julia> derivative(f, 2.0)

For symbolic derivatives, use Symbolics.jl.

I would say that ForwardDiff computes the exact symbolic derivative, though you can only directly access the result evaluated using floating-point arithmetic. If you look at the assembly code the compiler generates from ForwardDiff, it contains the exact symbolic derivative formula; for example, here it is computing that the derivative of x^2 is 2x:

julia> f(x) = x^2

julia> @code_llvm f(3.0)
define double @julia_f_202(double %0) {
    %1 = fmul double %0, %0
  ret double %1

julia> @code_llvm ForwardDiff.derivative(f, 3.0)
     %1 = fmul double %0, 2.000000e+00
  ret double %1

Whereas ‚Äúnumerical derivatives‚ÄĚ typically refer to approximate derivative formulas obtained from a sequence of function values only, such as those computed by FiniteDifferences.jl.


Thank you for the insight. According to this Wikipedia article, automatic differentiation as used in ForwardDiff is distinct from symbolic differentiation and numerical differentiation.

Btw, why the right-limit is output for the following derivative at zero?

using ForwardDiff
import ForwardDiff.derivative
f(x) = abs(x)

julia> derivative(f, 0)

And interestingly the left-limit can be obtained too:

julia> derivative(f, -0.0)

While strictly speaking, the derivative at 0 does not exist.

Thank you.


Most automatic differentiation systems take the point of view that they can essentially return anything at a non-differentiable point, and should prefer to return a value that is useful. There’s currently a ChainRulesCore.jl pull request to document this convention, here is a preview of the proposed documentation.