# Enzyme with Const() on a vector throws an error
**URL:** https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927
**Category:** Optimization (Mathematical)
**Tags:** autodiff, enzyme
**Created:** [October 29, 2024, 3:39pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927 "2024-10-29T15:39:48Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@weltenbummler](https://discourse.julialang.org/u/weltenbummler)
#### Post date: [October 29, 2024, 3:39pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/1 "2024-10-29T15:39:48Z")
Hello all,
I would like to autodiff a function where some vector-valued arguments are kept constant. I thought to use `Enzyme` with the `Const()` functionality but run into issues when applying `Const()` to vectors.
The following is a minimal working example that reproduces the problem. I use
Enzyme v0.13.12 with julia 1.11
```julia
using Enzyme
# this will work
function f(x::Array{Float64}, c::Vector{Float64})
y = (x[1]-c[1]) * (x[1]-c[1]) + (x[2]-c[2]) * (x[2]-c[2])
return y
end;
# this won't work
function h(x::Array{Float64}, c::Vector{Float64})
y = sum( (x-c).^2 )
return y
end;
# this will work
function h2(x::Array{Float64}, c1::Float64, c2::Float64)
c = [c1, c2]
y = sum( (x-c).^2 )
return y
end;
```
The three functions compute the squared norm between the two vectors `x` and `c`. For example
```julia
x = [4.0, 3.0];
c = [2.0, 1.0];
f(x, c)
f(x, c) == h(x, c) == h2(x, c[1], c[2]) # returns true
```
Now, `autodiff` on `f` and `h2` works
```julia
dx = [0.0, 0.0]
autodiff(Reverse, f, Active, Duplicated(x, dx), Const(c));
dx
2*(x-c) == dx # true
dx = [0.0, 0.0]
autodiff(Reverse, h2, Active, Duplicated(x, dx), Const(c[1]), Const(c[2]));
dx
2*(x-c) == dx # true
```
However, for `h` I get a `Constant memory is stored (or returned) to a differentiable variable.` error
```julia
dx = [0.0, 0.0]
autodiff(Reverse, h, Active, Duplicated(x, dx), Const(c));
dx
```
I am new to Julia and its AD system and puzzled by this error. It seems to me that `Const(c)` did not work when `c` is a vector? What would I need to change to make it work? Manually expanding the vector `c` into scalars won’t be an option for me.
Many thanks for your help.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [October 29, 2024, 8:38pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/2 "2024-10-29T20:38:20Z")
I can’t reproduce this on Julia 1.10.6 with Enzyme v0.13.12. Maybe you’re using an older version of Enzyme?
---
### Author:  [@yolhan\_mannes](https://discourse.julialang.org/u/yolhan_mannes)
#### Post date: [October 29, 2024, 8:50pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/3 "2024-10-29T20:50:18Z")
Enzyme on julia 1.11 has some issues still use it on Julia 1.10
---
### Author:  [@weltenbummler](https://discourse.julialang.org/u/weltenbummler)
#### Post date: [October 29, 2024, 11:15pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/4 "2024-10-29T23:15:20Z")
This seems to be indeed a Julia version issue. The posted code works with Julia 1.10.6, Enzyme v0.13.12 but not with Julia 1.11 (again with Enzyme v0.13.12).
---
### Author:  [@weltenbummler](https://discourse.julialang.org/u/weltenbummler)
#### Post date: [October 29, 2024, 11:17pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/5 "2024-10-29T23:17:55Z")
The issue seems to be with the julia version (v 1.11.1) and not the Enzyme version; I was using the same as you. Things indeed work with Julia v 1.10.6. Many thanks for the response.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [October 30, 2024, 5:07am UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/6 "2024-10-30T05:07:19Z")
Yeah Enzyme still has a few things to work out for v1.11
---
### Author:  [@wsmoses](https://discourse.julialang.org/u/wsmoses)
#### Post date: [November 5, 2024, 3:41pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/7 "2024-11-05T15:41:52Z")
After debugging with @gbaraldi @jameson and @Oscar_Smith, we have found a similar issue to be caused by a performance regression in 1.11.
I believe a fix is in the works ([Allow taking Matrix slices without an extra allocation by vtjnash · Pull Request #56236 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/56236)) and will be subsequently released in a patch of julia 1.11
In particular, 1.11’s new array implementation seems to have caused issues for alias analysis. In particular, [julia/base/abstractarray.jl at 9af0dea9d2c7c957bec7a14acacf0b234447be31 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/9af0dea9d2c7c957bec7a14acacf0b234447be31/base/abstractarray.jl#L1587) causes issues in proving a fresh allocation doesn’t alias with other data.
It’s likely the root cause of your issue above is something similar, so hopefully that fix gets pushed into 1.11 soon.
---
### Author:  [@wsmoses](https://discourse.julialang.org/u/wsmoses)
#### Post date: [November 6, 2024, 5:47pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/8 "2024-11-06T17:47:00Z")
A workaround has been made to the latest release which enables your case (and simple broadcasting) above to work on 1.11 without runtime activity.
---
### Author:  [@weltenbummler](https://discourse.julialang.org/u/weltenbummler)
#### Post date: [November 7, 2024, 12:03pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/9 "2024-11-07T12:03:36Z")
Awesome, thank you for letting me know!
---
### Author:  [@slwu89](https://discourse.julialang.org/u/slwu89)
#### Post date: [April 9, 2025, 2:46pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/10 "2025-04-09T14:46:23Z")
Sorry to revive the old thread but I came here via a google after the same problem (also new to AD in Julia). I’m on Julia v1.11.3 and the following package versions:
```julia
(julia) pkg> st
[a0c0ee7d] DifferentiationInterface v0.6.50
[7da242da] Enzyme v0.13.35
⌅ [f6369f11] ForwardDiff v0.10.38
```
Using Enzyme with `Const` wrapping a `DataFrame` gives the same error, is this expected, or is it fixed and I am somehow behind on versions (though I don’t appear to be)?
```julia
using RDatasets, DataFrames
using DifferentiationInterface
using Enzyme
using ForwardDiff
trees = dataset("datasets", "trees")
tree_mod_expectation = function(beta, data)
return @. beta[1] * (data[!, :Girth]^beta[2]) * (data[!, :Height]^beta[3])
end
beta = [.002, 2, 1]
f = tree_mod_expectation
DifferentiationInterface.jacobian(f, AutoEnzyme(), beta, Constant(trees))
DifferentiationInterface.jacobian(f, AutoForwardDiff(), beta, Constant(trees))
```
The error:
```julia
julia> DifferentiationInterface.jacobian(f, AutoEnzyme(), beta, Constant(trees))
ERROR: Constant memory is stored (or returned) to a differentiable variable.
As a result, Enzyme cannot provably ensure correctness and throws this error.
This might be due to the use of a constant variable as temporary storage for active memory (https://enzyme.mit.edu/julia/stable/faq/#Runtime-Activity).
If Enzyme should be able to prove this use non-differentable, open an issue!
To work around this issue, either:
a) rewrite this variable to not be conditionally active (fastest, but requires a code change), or
b) set the Enzyme mode to turn on runtime activity (e.g. autodiff(set_runtime_activity(Reverse), ...) ). This will maintain correctness, but may slightly reduce performance.
Mismatched activity for: ret {} addrspace(10)* %126, !dbg !285 const val: %126 = load atomic {} addrspace(10)*, {} addrspace(10)* addrspace(13)* %125 unordered, align 8, !dbg !296, !tbaa !272, !alias.scope !108, !noalias !111, !enzyme_type !271
Type tree: {}
llvalue= %125 = getelementptr inbounds {} addrspace(10)*, {} addrspace(10)* addrspace(13)* %124, i64 %109, !dbg !296
Stacktrace:
[1] getindex
@ ~/.julia/packages/DataFrames/kcA9R/src/dataframe/dataframe.jl:558
Stacktrace:
[1] lookupname
@ ~/.julia/packages/DataFrames/kcA9R/src/other/index.jl:0 [inlined]
[2] getindex
@ ~/.julia/packages/DataFrames/kcA9R/src/other/index.jl:440 [inlined]
[3] getindex
@ ~/.julia/packages/DataFrames/kcA9R/src/dataframe/dataframe.jl:557
[4] #11
@ ~/Desktop/git/FunWithSplines/julia/mgcv.jl:9 [inlined]
[5] fwddiffe3julia__11_29842wrap
@ ~/Desktop/git/FunWithSplines/julia/mgcv.jl:0
[6] macro expansion
@ ~/.julia/packages/Enzyme/g1jMR/src/compiler.jl:5445 [inlined]
[7] enzyme_call
@ ~/.julia/packages/Enzyme/g1jMR/src/compiler.jl:4983 [inlined]
[8] ForwardModeThunk
@ ~/.julia/packages/Enzyme/g1jMR/src/compiler.jl:4871 [inlined]
[9] autodiff
@ ~/.julia/packages/Enzyme/g1jMR/src/Enzyme.jl:654 [inlined]
[10] autodiff
@ ~/.julia/packages/Enzyme/g1jMR/src/Enzyme.jl:524 [inlined]
[11] macro expansion
@ ~/.julia/packages/Enzyme/g1jMR/src/sugar.jl:726 [inlined]
[12] #gradient#126
@ ~/.julia/packages/Enzyme/g1jMR/src/sugar.jl:582 [inlined]
[13] #jacobian#128
@ ~/.julia/packages/Enzyme/g1jMR/src/sugar.jl:789 [inlined]
[14] jacobian
@ ~/.julia/packages/Enzyme/g1jMR/src/sugar.jl:788 [inlined]
[15] jacobian(f::var"#11#12", prep::DifferentiationInterfaceEnzymeExt.EnzymeForwardOneArgJacobianPrep{…}, backend::AutoEnzyme{…}, x::Vector{…}, contexts::Constant{…})
@ DifferentiationInterfaceEnzymeExt ~/.julia/packages/DifferentiationInterface/7eD1K/ext/DifferentiationInterfaceEnzymeExt/forward_onearg.jl:231
[16] jacobian(f::var"#11#12", backend::AutoEnzyme{Nothing, Nothing}, x::Vector{Float64}, contexts::Constant{DataFrame})
@ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/7eD1K/src/fallbacks/no_prep.jl:51
[17] top-level scope
@ ~/Desktop/git/FunWithSplines/julia/mgcv.jl:16
```
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [April 9, 2025, 5:09pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/11 "2025-04-09T17:09:52Z")
Hi! Can you please try it with Enzyme’s native `jacobian` API for an MWE?
---
### Author:  [@slwu89](https://discourse.julialang.org/u/slwu89)
#### Post date: [April 9, 2025, 5:43pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/12 "2025-04-09T17:43:14Z")
No problem, this reproduces the error:
```julia
using RDatasets, DataFrames
using Enzyme
trees = dataset("datasets", "trees")
tree_mod_expectation = function(beta, data)
return @. beta[1] * (data[!, :Girth]^beta[2]) * (data[!, :Height]^beta[3])
end
beta = [.002, 2, 1]
Enzyme.jacobian(Forward, x -> tree_mod_expectation(x, trees), beta)
```
---
### Author:  [@wsmoses](https://discourse.julialang.org/u/wsmoses)
#### Post date: [April 9, 2025, 5:54pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/13 "2025-04-09T17:54:08Z")
Per the error message saying to mark the function as constant, what if you use:
```julia
Enzyme.jacobian(Forward, Const(x -> tree_mod_expectation(x, trees)), beta)
```
---
### Author:  [@wsmoses](https://discourse.julialang.org/u/wsmoses)
#### Post date: [April 9, 2025, 5:54pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/14 "2025-04-09T17:54:47Z")
Alternatively (and preferably),
```julia
Enzyme.jacobian(Forward, Const(tree_mod_expectation), x, Const(trees))
```
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [April 9, 2025, 6:38pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/15 "2025-04-09T18:38:06Z")
And if you’re already using DifferentiationInterface.jl in your code, you can force this function annotation by using
```julia
backend = AutoEnzyme(mode=Enzyme.Forward, function_annotation=Enzyme.Const)
```
---
### Author:  [@slwu89](https://discourse.julialang.org/u/slwu89)
#### Post date: [April 9, 2025, 7:53pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/16 "2025-04-09T19:53:37Z")
Thanks both @wsmoses and @gdalle.
My only understanding of what this constant marking is doing is from the docs here [Advanced tutorial · DifferentiationInterface.jl](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/tutorials/advanced/#Contexts), I did not know you can also mark an entire function with `Const`. Can you help point me to the docs where I can read more about what `Const(tree_mod_expectation)` means when applied to a function? Looking into the Enzyme docs I can only find [API reference · Enzyme.jl](https://enzyme.mit.edu/julia/stable/api/#EnzymeCore.Const) which explains what happens when function arguments are marked as `Const`.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [April 9, 2025, 8:04pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/17 "2025-04-09T20:04:47Z")
As you noticed, DifferentiationInterface.jl has an annotation system which is a bit similar to that of Enzyme.jl, but with different conventions in order to apply more or less equally to all autodiff backends. The first argument passed to the function `f` is always differentiated, and then you can annotate the rest with either `DI.Constant` (which translates to `Enzyme.Const`) or `DI.Cache` (which roughly translates to `Enzyme.Duplicated`).
The trouble is, Enzyme.jl also allows you to annotate the function itself, when it contains some data which may play a role in differentiation. This is such an advanced and unique aspect that I didn’t make room for it in DI’s native API. Instead, I chose to put that setting in the backend object `AutoEnzyme` (see its [docstring](https://docs.sciml.ai/ADTypes/stable/#ADTypes.AutoEnzyme)). So whenever you use `AutoEnzyme` with `function_annotation=Enzyme.Const`, you’re forcing DI to annotate `f` as `Enzyme.Const(f)` before calling `Enzyme.autodiff / Enzyme.jacobian / Enzyme.gradient`. Does that clarify things?
---
### Author:  [@wsmoses](https://discourse.julialang.org/u/wsmoses)
#### Post date: [April 9, 2025, 9:20pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/18 "2025-04-09T21:20:16Z")
Enzyme has no distinction between a function argument and any other argument in this sense that it can contain data to be marked differentiable or not.
You can mark any argument (or function argument) as Enzyme.Const to ask Enzyme not to differentiate with respect to it. See, for example: [API reference · Enzyme.jl](https://enzymead.github.io/Enzyme.jl/dev/api/#Enzyme.gradient-Union%7BTuple%7BN%7D,%20Tuple%7Bty_0%7D,%20Tuple%7BST%7D,%20Tuple%7BCS%7D,%20Tuple%7BRuntimeActivity%7D,%20Tuple%7BErrIfFuncWritten%7D,%20Tuple%7BABI%7D,%20Tuple%7BReturnPrimal%7D,%20Tuple%7BF%7D,%20Tuple%7BForwardMode%7BReturnPrimal,%20ABI,%20ErrIfFuncWritten,%20RuntimeActivity%7D,%20F,%20ty_0,%20Vararg%7BAny,%20N%7D)}}%20where%20{F,%20ReturnPrimal,%20ABI,%20ErrIfFuncWritten,%20RuntimeActivity,%20CS,%20ST,%20ty\_0,%20N}
This does carry with it the semantics that if you store differentiable data into a variable marked Const, and later load from it, you may get a counter-intuitive result (see [FAQ · Enzyme.jl](https://enzymead.github.io/Enzyme.jl/dev/faq/#Activity-of-temporary-storage)).
Though also looking more closely at the error message, it said:
```julia
ERROR: Constant memory is stored (or returned) to a differentiable variable.
As a result, Enzyme cannot provably ensure correctness and throws this error.
This might be due to the use of a constant variable as temporary storage for active memory (https://enzyme.mit.edu/julia/stable/faq/#Runtime-Activity).
If Enzyme should be able to prove this use non-differentable, open an issue!
To work around this issue, either:
a) rewrite this variable to not be conditionally active (fastest, but requires a code change), or
b) set the Enzyme mode to turn on runtime activity (e.g. autodiff(set_runtime_activity(Reverse), ...) ). This will maintain correctness, but may slightly reduce performance.
```
which indicates the issue is not a function argument (though its possible that would also resolve).
Here Enzyme is suggesting you try wrapping the Enzyme mode inside of a runtime activity, e.g. `Enzyme.autodiff(set_runtime_activity(Reverse), ...)`
In our case that would be
```julia
Enzyme.jacobian(set_runtime_activity(Forward), Const(tree_mod_expectation), x, Const(trees))
```
All Enzyme native functions (autodiff, gradient, jacobian) take a mode, which can take parameters, like runtime activity as here.
I’d generally recommend using the Enzyme native API as it is the only one officially supported (and also the error messages actually correspond to that API xD).
---
### Author:  [@slwu89](https://discourse.julialang.org/u/slwu89)
#### Post date: [April 9, 2025, 10:06pm UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/19 "2025-04-09T22:06:36Z")
Indeed it’s very clear, thanks!
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [April 10, 2025, 5:41am UTC](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927/20 "2025-04-10T05:41:54Z")
> [@wsmoses](#):
>
> All Enzyme native functions (autodiff, gradient, jacobian) take a mode, which can take parameters, like runtime activity as here.
If you want to reproduce this behavior in DI, again it goes through the backend because it is very specific to Enzyme:
```julia
backend = AutoEnzyme(mode=Enzyme.set_runtime_activity(Enzyme.Forward))
```
> [@wsmoses](#):
>
> I’d generally recommend using the Enzyme native API as it is the only one officially supported (and also the error messages actually correspond to that API xD).
Perhaps I could add error hints to DI for these cases, pointing people to the runtime activity and function annotation options inside `AutoEnzyme`? That way they could figure it out on their own, or know where to look in the docs?
Error hints are additional error messages that are displayed on top of a thrown exception, like this one for when a backend isn’t loaded and triggers a `MethodError`:
```julia
julia> using DifferentiationInterface
julia> gradient(sum, AutoEnzyme(), [1.0])
ERROR: MethodError: no method matching _prepare_pullback_aux(::Val{…}, ::DifferentiationInterface.PullbackFast, ::typeof(sum), ::AutoEnzyme{…}, ::Vector{…}, ::Tuple{…})
The function `_prepare_pullback_aux` exists, but no method is defined for this combination of argument types.
The autodiff backend you chose requires a package which may not be loaded. Please run the following command and try again:
import Enzyme
Closest candidates are:
_prepare_pullback_aux(::Val, ::DifferentiationInterface.PullbackSlow, ::F, ::ADTypes.AbstractADType, ::Any, ::NTuple{N, T} where {N, T}, Context...) where {F, C}
@ DifferentiationInterface ~/Documents/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/first_order/pullback.jl:157
_prepare_pullback_aux(::Val, ::DifferentiationInterface.PullbackSlow, ::F, ::Any, ::ADTypes.AbstractADType, ::Any, ::NTuple{N, T} where {N, T}, ::Context...) where {F, C}
@ DifferentiationInterface ~/Documents/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/src/first_order/pullback.jl:174
```
It’s not the same as a custom error message because I can’t control the creation of the `MethodError` here, so I just add something a posteriori after it has been thrown (see [this file](https://github.com/JuliaDiff/DifferentiationInterface.jl/blob/main/DifferentiationInterface/src/init.jl) for the definition of the hint).
If there is a well-defined and public type in Enzyme for this specific error (as in, `throw(EnzymeConstantDifferentiableError("blablabla"))` instead of `error("blablabla")`), I can catch it and add a short text telling users what to try?
[Next page](https://discourse.julialang.org/t/enzyme-with-const-on-a-vector-throws-an-error/121927.md?page=2)