# Enzyme.jl Critical Issues with Basic Arithmetic Operations in Automatic Differentiation

**URL:** <https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216>\
**Category:** Numerics\
**Tags:** autodiff, enzyme\
**Created:** [May 21, 2025, 1:11pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216 "2025-05-21T13:11:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mra](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mra](https://discourse.julialang.org/u/mra)\
**Post date:** [May 21, 2025, 1:11pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/1 "2025-05-21T13:11:06Z")

</div>

According to the Enzyme.jl documentation the “Enzyme differentiates arbitrary multivariate vector functions as the most general case in automatic differentiation”, however I encountered critical errors (including the kernel crash) just by using the basic arithmetic operations.

Here’s a minimal working example (MWE):

```julia
using Enzyme

foo1(x, y) = x[1] * y[1] + x[2] * y[2]
foo2(x, y) = x[1] * y[1] + x[2] * y[2] + 1.0
foo3(x, y) = x' * y
foo4(x, y) = x' * y + 1.0
foo5(x, y) = x .* y
foo6(x, y) = sum(x .* y)
foo7(x, y) = x + y
foo8(x, y) = sum(x + y)
foo9(x, y) = sum(foo7(x, y))
foo10(x, y) = [x[1] + y[1], x[2] + y[2]]
foo11(x, y) = x .+ y
foo12(x, y) = x .+ y .+ 1.0
foo13(x, y) = sum(foo12(x, y))

function test_func(func)
    x = [2.0, 3.0]
    y = [4.0, 1.0]

    grad_reverse_fail = nothing
    grad_forward_fail = nothing
    try 
        Enzyme.gradient(Reverse, func, x, y)
    catch e
        grad_reverse_fail = e
    end

    try 
        Enzyme.gradient(Forward, func, x, y)
    catch e
        grad_forward_fail = e
    end

    if grad_reverse_fail !== nothing
        println("Reverse gradient failed for $(func): ", grad_reverse_fail)
    else
        println("Reverse gradient succeeded for $(func)")
    end

    if grad_forward_fail !== nothing
        println("Forward gradient failed for $(func): ", grad_forward_fail)
    else
        println("Forward gradient succeeded for $(func)")
    end
end

test_func(foo1) # foo1 works
test_func(foo2) # foo2 works
test_func(foo3) # foo3 works
test_func(foo4) # foo4 works
test_func(foo5) # Reverse fails: Enzyme mutability error; Forward works
test_func(foo6) # foo6 works
test_func(foo7) # Reverse fails: Enzyme mutability error; Forward works
test_func(foo8) # foo8 works
test_func(foo9) # foo9 works
test_func(foo10) # Reverse fails: Enzyme mutability error; Forward works
test_func(foo11) # Reverse fails; Forward works
test_func(foo13) # Reverse fails; Forward fails: EnzymeRuntimeActivityError
test_func(foo12) # foo12 last, as it fails catastrophically and crashes the kernel

```

Interesting examples:  
foo7 and foo9, where foo9 calls foo7 and suddenly the differentiation works again (presumably the problem is that the function returns more than one argument; for context documentation says that Enzyme should work on any f: R^n → R^m)

foo12 and foo13, where foo12 terminates the julia kernel, while foo13 (which is calling foo12) fails, but does not crash the kernel.

Why are these simple examples failing? What can I realistically expect from Enzyme with these functionalities? I’m considering Enzyme for sensitivity analysis of differential equations - should I proceed with this package, or is it currently unreliable for such applications?

Thanks for your insights!

---

<div class="post-metadata">

**Author:** ![Craig\_Hamel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/craig_hamel/32/202434_2.png) [@Craig\_Hamel](https://discourse.julialang.org/u/Craig_Hamel)\
**Post date:** [May 21, 2025, 2:10pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/2 "2025-05-21T14:10:14Z")

</div>

In `foo5` and `foo7` you are mutating the memory. You need to use `Duplicated` in Enzyme to pass a mirror/shadow/not sure the right word of x and y. The same goes for your functions with `.` in them.

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [May 21, 2025, 2:36pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/3 "2025-05-21T14:36:57Z")

</div>

Enzyme.gradient assumes a scalar (not vector return), [Home · Enzyme.jl](https://enzymead.github.io/Enzyme.jl/dev/#Gradient-Convenience-functions:)

"Key convenience functions for common derivative computations are gradient (and its inplace variant gradient!). Like autodiff, the mode (forward or reverse) is determined by the first argument.

The functions gradient and gradient! compute the gradient of function with vector input and scalar return."

You should likely use Enzyme.jacobian for functions which return a vector, [Home · Enzyme.jl](https://enzymead.github.io/Enzyme.jl/dev/#Jacobian-Convenience-functions)

---

<div class="post-metadata">

**Author:** ![mra](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mra](https://discourse.julialang.org/u/mra)\
**Post date:** [May 23, 2025, 9:20am UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/4 "2025-05-23T09:20:21Z")

</div>

I have tried suggestions proposed by you (using Duplicated and jacobian) and I have also looked again through examples in documentation. Unfortunately, I still experience a lot of errors, which seem to be very unpredictable.

```julia
using Enzyme

foo1(x, y) = x[1] * y[1] + x[2] * y[2]
foo5(x, y) = x .* y
foo12(x, y) = x .+ y .+ 1.0
foo14(x, y, z) = begin
    z .= x .* y .+ 1.0
    return nothing
end

x = [2.0, 3.0]
y = [4.0, 1.0]

Enzyme.jacobian(Forward, foo1, x, y) # works
Enzyme.jacobian(Reverse, foo1, x, y) # fails, ERROR: MethodError: no method matching jacobian

dx, dy = zeros(size(x)), zeros(size(y))
Enzyme.autodiff(Forward, foo1, Duplicated(x, dx), Duplicated(y, dy)) # does not fail, but produces wrong result
println(dx, dy)

dx, dy = zeros(size(x)), zeros(size(y))
Enzyme.autodiff(Reverse, foo1, Duplicated(x, dx), Duplicated(y, dy)) # works
println(dx, dy)

Enzyme.jacobian(Forward, foo5, x, y) # works, but with warning, Warning: TODO forward zero-set of memorycopy used memset rather than runtime type
Enzyme.jacobian(Reverse, foo5, x, y) # fails, ERROR: MethodError: no method matching jacobian

dx, dy = zeros(size(x)), zeros(size(y))
Enzyme.autodiff(Forward, foo5, Duplicated(x, dx), Duplicated(y, dy)) # does not fail, but produces wrong result
println(dx, dy)

dx, dy = zeros(size(x)), zeros(size(y))
Enzyme.autodiff(Reverse, foo5, Duplicated(x, dx), Duplicated(y, dy)) # fails, ERROR: Duplicated Returns not yet handled
println(dx, dy)

z = zeros(size(x))
dx, dy, dz = zeros(size(x)), zeros(size(y)), ones(size(z))
Enzyme.autodiff(Forward, foo14, Duplicated(x, dx), Duplicated(y, dy), Duplicated(z, dz)) # does not fail, but produces wrong result
println(dx, dy, dz)

z = zeros(size(x))
dx, dy, dz = zeros(size(x)), zeros(size(y)), ones(size(z))
Enzyme.autodiff(Reverse, foo14, Duplicated(x, dx), Duplicated(y, dy), Duplicated(z, dz)) # works
println(dx, dy, dz)

Enzyme.jacobian(Reverse, foo12, x, y) # fails, ERROR: MethodError: no method matching jacobian
Enzyme.jacobian(Forward, foo12, x, y) # crashes the kernel

```

1. What is the correct way to compute derivatives for such functions in both modes?
2. Why `Forward` and `Reverse` produce different results even if they are called in the same way?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 23, 2025, 9:52am UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/5 "2025-05-23T09:52:02Z")

</div>

Most of these errors are due to the way you’re (mis)using the Enzyme.jl API.

> [@mra](#):
>
> `Enzyme.jacobian(Reverse, foo1, x, y) # fails, ERROR: MethodError: no method matching jacobian`

> [@mra](#):
>
> `Enzyme.jacobian(Reverse, foo5, x, y) # fails, ERROR: MethodError: no method matching jacobian`

Reverse-mode `jacobian` [does not support](https://enzymead.github.io/Enzyme.jl/stable/api/#Enzyme.jacobian-Union%7BTuple%7BHolomorphic%7D,%20Tuple%7BCT%7D,%20Tuple%7BOutType%7D,%20Tuple%7BRuntimeActivity%7D,%20Tuple%7BErrIfFuncWritten%7D,%20Tuple%7BRABI%7D,%20Tuple%7BX%7D,%20Tuple%7BF%7D,%20Tuple%7BReturnPrimal%7D,%20Tuple%7BReverseMode%7BReturnPrimal,%20RuntimeActivity,%20RABI,%20Holomorphic,%20ErrIfFuncWritten%7D,%20F,%20X) several arguments.

> [@mra](#):
>
> `Enzyme.autodiff(Forward, foo1, Duplicated(x, dx), Duplicated(y, dy)) # does not fail, but produces wrong result`

> [@mra](#):
>
> `Enzyme.autodiff(Forward, foo5, Duplicated(x, dx), Duplicated(y, dy)) # does not fail, but produces wrong result`

Forward-mode `autodiff` requires you to specify input perturbations that are [propagated](https://enzymead.github.io/Enzyme.jl/stable/#Forward-mode) to the output. Here the input perturbations are all zero, so no derivatives are propagated.

> [@mra](#):
>
> `Enzyme.autodiff(Reverse, foo5, Duplicated(x, dx), Duplicated(y, dy)) # fails, ERROR: Duplicated Returns not yet handled`

See Billy’s remark above, for vector returns you need to reformulate your function `f(x) = y` as `f!(y, x) = nothing`.

> [@wsmoses](#):
>
> Enzyme.gradient assumes a scalar (not vector return), [Home · Enzyme.jl](https://enzymead.github.io/Enzyme.jl/dev/#Gradient-Convenience-functions:)

* * *

If you are struggling with the Enzyme.jl API, you may find it useful to try DifferentiationInterface.jl as a starting point. It is much less powerful and only supports a single active argument, but it hides the complexity of handling activity annotations (`Duplicated` and friends) from the user. In particular, `DI.pushforward` and `DI.pullback` are useful to compute JVPs and VJPs respectively.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 23, 2025, 10:19am UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/6 "2025-05-23T10:19:10Z")

</div>

To get your mind around the way forward and reverse modes work in Enzyme, this discussion might help:

> [@Do I understand Enzyme properly?](https://discourse.julialang.org/t/do-i-understand-enzyme-properly/97760):
>
> I think I finally have a grasp on how Enzyme works, but I would like an expert to validate this. Consider a possibly mutating vector function f(x, y) which outputs a scalar value z. We denote by x\_a and y\_a the contents of x and y after execution. Enzyme works with a Jacobian including all of these variables (defined by blocks): J = \begin{pmatrix} \partial x\_a / \partial x & \partial x\_a / \partial y \\ \partial y\_a / \partial x & \partial y\_a / \partial y \\ \partial z / \partial x & \partia…

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [May 23, 2025, 1:57pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/7 "2025-05-23T13:57:06Z")

</div>

I don’t think the Duplicated comment was correct, your use of gradient was correct for scalar returns.

Indeed reverse-mode jacobians doesn’t support multiple arguments at the moment, that can be resolved as follows.

I don’t get the crash you see – can you open an issue with yout Enzyme version, Julia version, OS details, etc.

```julia
(base) wmoses@MacBook-Pro-18 Enzyme.jl % julia --project
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.10.9 (2025-03-10)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

julia> using Enzyme

julia> foo1(x, y) = x[1] * y[1] + x[2] * y[2]
foo1 (generic function with 1 method)

julia> foo5(x, y) = x .* y
foo5 (generic function with 1 method)

julia> foo12(x, y) = x .+ y .+ 1.0
foo12 (generic function with 1 method)

julia> foo14(x, y, z) = begin
           z .= x .* y .+ 1.0
           return nothing
       end
foo14 (generic function with 1 method)

julia> x = [2.0, 3.0]
2-element Vector{Float64}:
 2.0
 3.0

julia> y = [4.0, 1.0]
2-element Vector{Float64}:
 4.0
 1.0

julia> Enzyme.gradient(Forward, foo1, x, y)
([4.0, 1.0], [2.0, 3.0])

julia> Enzyme.gradient(Reverse, foo1, x, y)
([4.0, 1.0], [2.0, 3.0])

julia> Enzyme.jacobian(Forward, foo5, x, y)
([4.0 0.0; 0.0 1.0], [2.0 0.0; 0.0 3.0])

julia> (Enzyme.jacobian(Reverse, Base.Fix2(foo5, y), x)[1], Enzyme.jacobian(Reverse, Base.Fix1(foo5, x), y)[1])

       # Gradient assumes scalar return / jacobian assues vector return, your function returns values in the first argument we either call autodiff directly and pass shadows, or make the function
       # match the expected calling convention
([4.0 0.0; 0.0 1.0], [2.0 0.0; 0.0 3.0])

julia> function foo14_out(x, y)
           z = similar(x)
           foo14(z, x, y)
           return z
       end
foo14_out (generic function with 1 method)

julia> Enzyme.jacobian(Forward, foo14_out, x, y)
([6.551898133e-314 0.0; 7.746824126402542e-304 2.5904922037e-314], [0.0 6.543788026e-314; 2.556857416e-314 2.2234624005e-314])

julia> (Enzyme.jacobian(Reverse, Base.Fix2(foo14_out, y), x)[1], Enzyme.jacobian(Reverse, Base.Fix1(foo14_out, x), y)[1])

([0.0 0.0; 0.0 0.0], [0.0 0.0; 0.0 0.0])

julia> (Enzyme.jacobian(Reverse, Base.Fix2(foo12, y), x)[1], Enzyme.jacobian(Reverse, Base.Fix1(foo12, x), y)[1])
([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0])

julia> Enzyme.jacobian(Forward, foo12, x, y) # crashes the kernel
([1.0 0.0; 0.0 1.0], [1.0 0.0; 0.0 1.0])

```

---

<div class="post-metadata">

**Author:** ![mra](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@mra](https://discourse.julialang.org/u/mra)\
**Post date:** [May 26, 2025, 12:59pm UTC](https://discourse.julialang.org/t/enzyme-jl-critical-issues-with-basic-arithmetic-operations-in-automatic-differentiation/129216/8 "2025-05-26T12:59:59Z")

</div>

I have submitted the issue ( [Enzyme crashes Julia kernel on very simple example · Issue #2414 · EnzymeAD/Enzyme.jl](https://github.com/EnzymeAD/Enzyme.jl/issues/2414) ).