Zygote : Can't differentiate gc_preserve_end expression

General Usage
1

An error was reported when I called Zygote.gradient, with the following error message. What should I do to resolve it?

image
image850×683 21.2 KB

1 Like
2

In my function myfunc, TensorOperations.jl is used to do the tensor contraction

1 Like
3

I don’t think TensorOperations.jl currently supports Zygote. You might be interested in GitHub - mcabbott/Tullio.jl: ⅀ though, which does support Zygote.

1 Like
5

I can not add this package, and, I dont know how to solve this problem.

  • The error message is as follows:

Updating registry at C:\Users\38606\.juliapro\JuliaPro_v1.2.0-2\registries\JuliaPro
Updating git-repo https://pkg.juliacomputing.com//registry/JuliaPro
Resolving package versions…
Unsatisfiable requirements detected for package Requires [ae029012]:
Requires [ae029012] log:
├─possible versions are: [0.5.0-0.5.2, 1.0.0-1.0.1] or uninstalled
├─restricted by compatibility requirements with Atom [c52e3926] to versions: 0.5.0-0.5.2
│ └─Atom [c52e3926] log:
│ ├─possible versions are: [0.8.0-0.8.8, 0.9.0-0.9.1, 0.10.0-0.10.2, 0.11.0-0.11.3, 0.12.0-0.12.19] or uninstalled
│ └─restricted to versions 0.11.3 by an explicit requirement, leaving only versions 0.11.3
└─restricted by compatibility requirements with Tullio [bc48ee85] to versions: 1.0.0-1.0.1 — no versions left
└─Tullio [bc48ee85] log:
├─possible versions are: [0.1.0-0.1.3, 0.2.0] or uninstalled
└─restricted to versions * by an explicit requirement, leaving only versions [0.1.0-0.1.3, 0.2.0]

Stacktrace:
[1] #propagate_constraints!#61(::Bool, ::typeof(Pkg.GraphType.propagate_constraints!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1005
[2] propagate_constraints! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:946 [inlined]
[3] #simplify_graph!#121(::Bool, ::typeof(Pkg.GraphType.simplify_graph!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460
[4] simplify_graph! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460 [inlined]
[5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:388
[6] resolve_versions! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:328 [inlined]
[7] #add_or_develop#63(::Array{Base.UUID,1}, ::Symbol, ::typeof(Pkg.Operations.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:1235
[8] #add_or_develop at .\none:0 [inlined]
[9] #add_or_develop#17(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:59
[10] (::getfield(Pkg.API, Symbol(“#kw##add_or_develop”)))(::NamedTuple{(:mode,),Tuple{Symbol}}, ::typeof(Pkg.API.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at .\none:0
[11] do_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:672
[12] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:790
[13] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:789
[14] do_cmd!(::Pkg.REPLMode.PkgCommand, ::IJulia.MiniREPL) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:563
[15] #do_cmd#31(::Bool, ::typeof(Pkg.REPLMode.do_cmd), ::IJulia.MiniREPL, ::String) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:538
[16] (::getfield(Pkg.REPLMode, Symbol(“#kw##do_cmd”)))(::NamedTuple{(:do_rethrow,),Tuple{Bool}}, ::typeof(Pkg.REPLMode.do_cmd), ::IJulia.MiniREPL, ::String) at .\none:0
[17] top-level scope at In[1]:1

6

You seem to have quite an old version of Atom holding it back. I’d suggest trying it with the official non-JuliaPro binaries.

7

May I ask how to achieve the contraction operation of tensors with Tullio.jl package?
I used the TensorOperations. jl package to achieve the tensor contraction as
@tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]);
but，I used the Tullio. jl package to achieve the tensor contraction as
@tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]);
which is wrong.

8

Can you explain what you mean by wrong? Once both packages are loaded, Tullio literally calls TensorOperations to do the work. For example the code printed by the first line here should be a subset of what’s printed by the second line:

@macroexpand @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])
@macroexpand @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])

Both contain things like TensorOperations.contract!(..., (2, 3), (1, 4), (2, 3), (1, 4), (1, 3), (2, 4), ...). But the second has a lot more below, to calculate the gradient for Zygote.

1 Like
9

The result calculated with @tullio is different from @ tensor,and then, when I use Zygote.gradient(), an error also occurs which say “Can’t differentiate foreigncall expression”

10

It’s hard to guess what’s going wrong, can you boil this down to a minimal example which runs by itself (or ought to)? What version of Tullio are you using?

For me these all agree, as they must:

T = randn(3,3,3,3);

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T)[1] # gradient calculated by @tullio & evaluated by @tensor

ForwardDiff.gradient(T -> sum(sin, @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T) # not using @tullio at all

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]) tensor=false avx=false), T)[1] # not using @tensor at all
1 Like
11

Thank you for your helpful response. One of my previous pieces of code was
@tullio T[w,x,z,y] := T[x,a,o,b] * U_l[a,A,z] * T[o,A,y,B] * U_l[b,B,w];
This leads to local variable name “T” conflicts with an argument. Now I have modified it and my function gives the same answer. But when I used Zygote.gradient(), it occurs an new error:
Mutating arrays is not supported

Stacktrace:
[1] error(::String) at .\error.jl:33
[2] (::Zygote.var"#451#452")(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\lib\array.jl:65
[3] (::Zygote.var"#2373#back#453"{Zygote.var"#451#452"})(::Nothing) at C:\Users\38606.julia\packages\ZygoteRules\6nssF\src\adjoint.jl:49
[4] Square_T0 at .\In[2]:13 [inlined]
[5] (::typeof(∂(Square_T0)))(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0
[6] hotrg at .\In[13]:5 [inlined]
[7] (::typeof(∂(hotrg)))(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0
[8] #70 at .\In[19]:1 [inlined]
[9] (::Zygote.var"#41#42"{typeof(∂(#70))})(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:45
[10] gradient(::Function, ::Array{Complex{Float64},2}, ::Vararg{Array{Complex{Float64},2},N} where N) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:54
[11] top-level scope at In[19]:1