Zygote : Can't differentiate gc_preserve_end expression

F-YF · August 15, 2020, 2:13am

An error was reported when I called Zygote.gradient, with the following error message. What should I do to resolve it?

F-YF · August 15, 2020, 2:16am

In my function myfunc, TensorOperations.jl is used to do the tensor contraction

simeonschaub · August 15, 2020, 8:55am

I don’t think TensorOperations.jl currently supports Zygote. You might be interested in GitHub - mcabbott/Tullio.jl: ⅀ though, which does support Zygote.

F-YF · August 16, 2020, 2:24pm

I can not add this package, and, I dont know how to solve this problem.

The error message is as follows:

Updating registry at C:\Users\38606\.juliapro\JuliaPro_v1.2.0-2\registries\JuliaPro
Updating git-repo https://pkg.juliacomputing.com//registry/JuliaPro
Resolving package versions…
Unsatisfiable requirements detected for package Requires [ae029012]:
Requires [ae029012] log:
├─possible versions are: [0.5.0-0.5.2, 1.0.0-1.0.1] or uninstalled
├─restricted by compatibility requirements with Atom [c52e3926] to versions: 0.5.0-0.5.2
│ └─Atom [c52e3926] log:
│ ├─possible versions are: [0.8.0-0.8.8, 0.9.0-0.9.1, 0.10.0-0.10.2, 0.11.0-0.11.3, 0.12.0-0.12.19] or uninstalled
│ └─restricted to versions 0.11.3 by an explicit requirement, leaving only versions 0.11.3
└─restricted by compatibility requirements with Tullio [bc48ee85] to versions: 1.0.0-1.0.1 — no versions left
└─Tullio [bc48ee85] log:
├─possible versions are: [0.1.0-0.1.3, 0.2.0] or uninstalled
└─restricted to versions * by an explicit requirement, leaving only versions [0.1.0-0.1.3, 0.2.0]

Stacktrace:
[1] #propagate_constraints!#61(::Bool, ::typeof(Pkg.GraphType.propagate_constraints!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1005
[2] propagate_constraints! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:946 [inlined]
[3] #simplify_graph!#121(::Bool, ::typeof(Pkg.GraphType.simplify_graph!), ::Pkg.GraphType.Graph, ::Set{Int64}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460
[4] simplify_graph! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\GraphType.jl:1460 [inlined]
[5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:388
[6] resolve_versions! at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:328 [inlined]
[7] #add_or_develop#63(::Array{Base.UUID,1}, ::Symbol, ::typeof(Pkg.Operations.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:1235
[8] #add_or_develop at .\none:0 [inlined]
[9] #add_or_develop#17(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\API.jl:59
[10] (::getfield(Pkg.API, Symbol(“#kw##add_or_develop”)))(::NamedTuple{(:mode,),Tuple{Symbol}}, ::typeof(Pkg.API.add_or_develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at .\none:0
[11] do_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:672
[12] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:790
[13] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:789
[14] do_cmd!(::Pkg.REPLMode.PkgCommand, ::IJulia.MiniREPL) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:563
[15] #do_cmd#31(::Bool, ::typeof(Pkg.REPLMode.do_cmd), ::IJulia.MiniREPL, ::String) at C:\Users\julia\AppData\Local\Julia-1.2.0\share\julia\stdlib\v1.2\Pkg\src\REPLMode.jl:538
[16] (::getfield(Pkg.REPLMode, Symbol(“#kw##do_cmd”)))(::NamedTuple{(:do_rethrow,),Tuple{Bool}}, ::typeof(Pkg.REPLMode.do_cmd), ::IJulia.MiniREPL, ::String) at .\none:0
[17] top-level scope at In[1]:1

simeonschaub · August 16, 2020, 3:05pm

You seem to have quite an old version of Atom holding it back. I’d suggest trying it with the official non-JuliaPro binaries.

F-YF · August 17, 2020, 10:29am

May I ask how to achieve the contraction operation of tensors with Tullio.jl package?
I used the TensorOperations. jl package to achieve the tensor contraction as
@tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]);
but，I used the Tullio. jl package to achieve the tensor contraction as
@tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]);
which is wrong.

mcabbott · August 17, 2020, 11:21am

Can you explain what you mean by wrong? Once both packages are loaded, Tullio literally calls TensorOperations to do the work. For example the code printed by the first line here should be a subset of what’s printed by the second line:

@macroexpand @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])
@macroexpand @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])

Both contain things like TensorOperations.contract!(..., (2, 3), (1, 4), (2, 3), (1, 4), (1, 3), (2, 4), ...). But the second has a lot more below, to calculate the gradient for Zygote.

F-YF · August 17, 2020, 1:07pm

The result calculated with @tullio is different from @ tensor,and then, when I use Zygote.gradient(), an error also occurs which say “Can’t differentiate foreigncall expression”

mcabbott · August 17, 2020, 1:22pm

It’s hard to guess what’s going wrong, can you boil this down to a minimal example which runs by itself (or ought to)? What version of Tullio are you using?

For me these all agree, as they must:

T = randn(3,3,3,3);

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T)[1] # gradient calculated by @tullio & evaluated by @tensor

ForwardDiff.gradient(T -> sum(sin, @tensor M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B])), T) # not using @tullio at all

Zygote.gradient(T -> sum(sin, @tullio M_M[a,A,c,C] := (T[x,a,z,b] * T[x,c,w,b])* (T[w,C,y,B] * T[z,A,y,B]) tensor=false avx=false), T)[1] # not using @tensor at all

F-YF · August 18, 2020, 2:25am

Thank you for your helpful response. One of my previous pieces of code was
@tullio T[w,x,z,y] := T[x,a,o,b] * U_l[a,A,z] * T[o,A,y,B] * U_l[b,B,w];
This leads to local variable name “T” conflicts with an argument. Now I have modified it and my function gives the same answer. But when I used Zygote.gradient(), it occurs an new error:
Mutating arrays is not supported

Stacktrace:
[1] error(::String) at .\error.jl:33
[2] (::Zygote.var"#451#452")(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\lib\array.jl:65
[3] (::Zygote.var"#2373#back#453"{Zygote.var"#451#452"})(::Nothing) at C:\Users\38606.julia\packages\ZygoteRules\6nssF\src\adjoint.jl:49
[4] Square_T0 at .\In[2]:13 [inlined]
[5] (::typeof(∂(Square_T0)))(::Nothing) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0
[6] hotrg at .\In[13]:5 [inlined]
[7] (::typeof(∂(hotrg)))(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface2.jl:0
[8] #70 at .\In[19]:1 [inlined]
[9] (::Zygote.var"#41#42"{typeof(∂(#70))})(::Float64) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:45
[10] gradient(::Function, ::Array{Complex{Float64},2}, ::Vararg{Array{Complex{Float64},2},N} where N) at C:\Users\38606.julia\packages\Zygote\seGHk\src\compiler\interface.jl:54
[11] top-level scope at In[19]:1