# Zygote : Can't differentiate gc\_preserve\_end expression

**URL:** <https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973>\
**Category:** General Usage\
**Created:** [August 15, 2020, 2:13am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973 "2020-08-15T02:13:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 15, 2020, 2:13am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/1 "2020-08-15T02:13:09Z")

</div>

An error was reported when I called Zygote.gradient, with the following error message. What should I do to resolve it?

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/c/bccc74f5a13e27f296877fb1a985118173f3a7d2.png)

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 15, 2020, 2:16am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/2 "2020-08-15T02:16:02Z")

</div>

In my function myfunc, TensorOperations.jl is used to do the tensor contraction

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [August 15, 2020, 8:55am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/3 "2020-08-15T08:55:00Z")

</div>

I don’t think TensorOperations.jl currently supports Zygote. You might be interested in [GitHub - mcabbott/Tullio.jl: ⅀](https://github.com/mcabbott/Tullio.jl) though, which does support Zygote.

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 16, 2020, 2:24pm UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/5 "2020-08-16T14:24:51Z")

</div>

I can not add this package, and, I dont know how to solve this problem.

- The error message is as follows:

Updating registry at `C:\Users\38606\.juliapro\JuliaPro_v1.2.0-2\registries\JuliaPro`  
Updating git-repo `https://pkg.juliacomputing.com//registry/JuliaPro`  
Resolving package versions…  
Unsatisfiable requirements detected for package Requires [ae029012]:  
Requires [ae029012] log:  
├─possible versions are: [0.5.0-0.5.2, 1.0.0-1.0.1] or uninstalled  
├─restricted by compatibility requirements with Atom [c52e3926] to versions: 0.5.0-0.5.2  
│ └─Atom [c52e3926] log:  
│ ├─possible versions are: [0.8.0-0.8.8, 0.9.0-0.9.1, 0.10.0-0.10.2, 0.11.0-0.11.3, 0.12.0-0.12.19] or uninstalled  
│ └─restricted to versions 0.11.3 by an explicit requirement, leaving only versions 0.11.3  
└─restricted by compatibility requirements with Tullio [bc48ee85] to versions: 1.0.0-1.0.1 — no versions left  
└─Tullio [bc48ee85] log:  
├─possible versions are: [0.1.0-0.1.3, 0.2.0] or uninstalled  
└─restricted to versions \* by an explicit requirement, leaving only versions [0.1.0-0.1.3, 0.2.0]

Stacktrace:  
[1] #propagate\_constraints!#61(::Bool, ::typeof(Pkg.GraphType.propagate\_constraints!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1005  
[2] propagate\_constraints! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:946 [inlined]  
[3] #simplify\_graph!#121(::Bool, ::typeof(Pkg.GraphType.simplify\_graph!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460  
[4] simplify\_graph! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460 [inlined]  
[5] resolve\_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:388  
[6] resolve\_versions! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:328 [inlined]  
[7] #add\_or\_develop#63(::Array{Base.UUID,1}, ::Symbol, ::typeof(Pkg.Operations.add\_or\_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:1235  
[8] #add\_or\_develop at .\none:0 [inlined]  
[9] #add\_or\_develop#17(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add\_or\_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:59  
[10] (::getfield(Pkg.API, Symbol(“#kw##add\_or\_develop”)))(::NamedTuple{(:mode,),Tuple{Symbol}}, ::typeof(Pkg.API.add\_or\_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at .\none:0  
[11] do\_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:672  
[12] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:790  
[13] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:789  
[14] do\_cmd!(::Pkg.REPLMode.PkgCommand, ::IJulia.MiniREPL) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:563  
[15] #do\_cmd#31(::Bool, ::typeof(Pkg.REPLMode.do\_cmd), ::IJulia.MiniREPL, ::String) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:538  
[16] (::getfield(Pkg.REPLMode, Symbol(“#kw##do\_cmd”)))(::NamedTuple{(:do\_rethrow,),Tuple{Bool}}, ::typeof(Pkg.REPLMode.do\_cmd), ::IJulia.MiniREPL, ::String) at .\none:0  
[17] top-level scope at In[1]:1

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [August 16, 2020, 3:05pm UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/6 "2020-08-16T15:05:31Z")

</div>

You seem to have quite an old version of `Atom` holding it back. I’d suggest trying it with the official non-JuliaPro binaries.

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 17, 2020, 10:29am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/7 "2020-08-17T10:29:51Z")

</div>

May I ask how to achieve the contraction operation of tensors with Tullio.jl package?  
I used the TensorOperations. jl package to achieve the tensor contraction as  
@tensor M\_M[a,A,c,C] := (T[x,a,z,b] \* T[x,c,w,b])\* (T[w,C,y,B] \* T[z,A,y,B]);  
but，I used the Tullio. jl package to achieve the tensor contraction as  
@tullio M\_M[a,A,c,C] := (T[x,a,z,b] \* T[x,c,w,b])\* (T[w,C,y,B] \* T[z,A,y,B]);  
which is wrong.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [August 17, 2020, 11:21am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/8 "2020-08-17T11:21:52Z")

</div>

Can you explain what you mean by wrong? Once both packages are loaded, Tullio literally calls TensorOperations to do the work. For example the code printed by the first line here should be a subset of what’s printed by the second line:

```julia
@macroexpand @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])
@macroexpand @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])

```

Both contain things like `TensorOperations.contract!(..., (2, 3), (1, 4), (2, 3), (1, 4), (1, 3), (2, 4), ...)`. But the second has a lot more below, to calculate the gradient for Zygote.

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 17, 2020, 1:07pm UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/9 "2020-08-17T13:07:26Z")

</div>

The result calculated with @tullio is different from @ tensor,and then, when I use Zygote.gradient(), an error also occurs which say “Can’t differentiate foreigncall expression”

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [August 17, 2020, 1:22pm UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/10 "2020-08-17T13:22:13Z")

</div>

It’s hard to guess what’s going wrong, can you boil this down to a minimal example which runs by itself (or ought to)? What version of Tullio are you using?

For me these all agree, as they must:

```julia
T = randn(3,3,3,3);

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T)[1] # gradient calculated by @tullio & evaluated by @tensor

ForwardDiff.gradient(T -> sum(sin, @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T) # not using @tullio at all

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]) tensor=false avx=false), T)[1] # not using @tensor at all

```

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [August 18, 2020, 2:25am UTC](https://discourse.julialang.org/t/zygote-cant-differentiate-gc-preserve-end-expression/44973/11 "2020-08-18T02:25:57Z")

</div>

Thank you for your helpful response. One of my previous pieces of code was  
@tullio T[w,x,z,y] := T[x,a,o,b] \* U\_l[a,A,z] \* T[o,A,y,B] \* U\_l[b,B,w];  
This leads to local variable name “T” conflicts with an argument. Now I have modified it and my function gives the same answer. But when I used Zygote.gradient(), it occurs an new error:  
Mutating arrays is not supported

Stacktrace:  
[1] error(::String) at .\error.jl:33  
[2] (::Zygote.var"#451#452")(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\lib\array.jl:65  
[3] (::Zygote.var"#2373#back#453"{Zygote.var"#451#452"})(::Nothing) at C:\Users\38606.julia\packages\ZygoteRules\6nssF\src\adjoint.jl:49  
[4] Square\_T0 at .\In[2]:13 [inlined]  
[5] (::typeof(∂(Square\_T0)))(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0  
[6] hotrg at .\In[13]:5 [inlined]  
[7] (::typeof(∂(hotrg)))(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0  
[8] #70 at .\In[19]:1 [inlined]  
[9] (::Zygote.var"#41#42"{typeof(∂(#70))})(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:45  
[10] gradient(::Function, ::Array{Complex{Float64},2}, ::Vararg{Array{Complex{Float64},2},N} where N) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:54  
[11] top-level scope at In[19]:1