How to pass the output of custom layers into Dense layer

Specific Domains Machine Learning
1

I am trying to do the following example:

using Flux

encoderA = Dense(10, 2)
encoderB(x) = atan(x[2], x[1])
decoderA(x) = [[cos(x)], [sin(x)]]
decoderB = Dense(2, 10)

m1 = Chain(encoderA, encoderB, decoderA, decoderB)

test = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
test1 = mapslices(x -> [x], test, dims=1)[:]


loss(x) = Flux.mse(m1(x), x)

Flux.@epochs 10 Flux.train!(loss, Flux.params(m1), zip(test1), ADAM(), cb = () -> println(loss(test)))

and get the following error

ERROR: LoadError: MethodError: no method matching +(::Array{Tracker.TrackedReal{Float32},1}, ::Float32)
Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...) at operators.jl:502
  +(::Bool, ::T<:AbstractFloat) where T<:AbstractFloat at bool.jl:112
  +(::Float32, ::Float32) at float.jl:394
  ...
Stacktrace:
 [1] _broadcast_getindex at ./broadcast.jl:352 [inlined]
 [2] getindex at ./broadcast.jl:511 [inlined]
 [3] copy at ./broadcast.jl:787 [inlined]
 [4] materialize at ./broadcast.jl:753 [inlined]
 [5] broadcast(::getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##1#3")),getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(+)},typeof(identity)}, ::Array{Array{Tracker.TrackedReal{Float32},1},1}, ::Array{Float32,1}) at ./broadcast.jl:707
 [6] ∇broadcast(::getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##1#3")),getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(+)},typeof(identity)}, ::TrackedArray{Array{Tracker.TrackedReal{Float32},1},1,Array{Array{Tracker.TrackedReal{Float32},1},1}}, ::TrackedArray{Float32,1,Array{Float32,1}}) at /Users/nicolasbarra/.julia/packages/Tracker/RRYy6/src/lib/array.jl:484
 [7] copy(::Base.Broadcast.Broadcasted{Tracker.TrackedStyle,Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{Tracker.TrackedStyle,Nothing,typeof(+),Tuple{TrackedArray{Array{Tracker.TrackedReal{Float32},1},1,Array{Array{Tracker.TrackedReal{Float32},1},1}},TrackedArray{Float32,1,Array{Float32,1}}}}}}) at /Users/nicolasbarra/.julia/packages/Tracker/RRYy6/src/lib/array.jl:515
 [8] materialize at ./broadcast.jl:753 [inlined]
 [9] Dense at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/layers/basic.jl:99 [inlined]
 [10] applychain at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/layers/basic.jl:31 [inlined] (repeats 4 times)
 [11] (::Chain{Tuple{Dense{typeof(identity),TrackedArray{Float32,2,Array{Float32,2}},TrackedArray{Float32,1,Array{Float32,1}}},typeof(encoderB),typeof(decoderA),Dense{typeof(identity),TrackedArray{Float32,2,Array{Float32,2}},TrackedArray{Float32,1,Array{Float32,1}}}}})(::Array{Int64,1}) at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/layers/basic.jl:33
 [12] loss(::Array{Int64,1}) at ./untitled-d82908f7118cb2d7d8caf09ecc08fd4c:14
 [13] #14 at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/optimise/train.jl:72 [inlined]
 [14] gradient_(::getfield(Flux.Optimise, Symbol("##14#20")){typeof(loss)}, ::Tracker.Params) at /Users/nicolasbarra/.julia/packages/Tracker/RRYy6/src/back.jl:97
 [15] #gradient#24(::Bool, ::Function, ::Function, ::Tracker.Params) at /Users/nicolasbarra/.julia/packages/Tracker/RRYy6/src/back.jl:164
 [16] gradient at /Users/nicolasbarra/.julia/packages/Tracker/RRYy6/src/back.jl:164 [inlined]
 [17] macro expansion at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/optimise/train.jl:71 [inlined]
 [18] macro expansion at /Users/nicolasbarra/.julia/packages/Juno/TfNYn/src/progress.jl:124 [inlined]
 [19] #train!#12(::getfield(Main, Symbol("##12#16")), ::Function, ::Function, ::Tracker.Params, ::Base.Iterators.Zip{Tuple{Array{Array{Int64,1},1}}}, ::ADAM) at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/optimise/train.jl:69
 [20] (::getfield(Flux.Optimise, Symbol("#kw##train!")))(::NamedTuple{(:cb,),Tuple{getfield(Main, Symbol("##12#16"))}}, ::typeof(Flux.Optimise.train!), ::Function, ::Tracker.Params, ::Base.Iterators.Zip{Tuple{Array{Array{Int64,1},1}}}, ::ADAM) at ./none:0
 [21] top-level scope at /Users/nicolasbarra/.julia/packages/Flux/qXNjB/src/optimise/train.jl:106
 [22] top-level scope at /Users/nicolasbarra/.julia/packages/Juno/TfNYn/src/progress.jl:124
in expression starting at untitled-d82908f7118cb2d7d8caf09ecc08fd4c:16

This is likely because the output of the first dense layer is
Tracked 2-element Array{Float32,1}:
and the output of the decoderA layer is
2-element Array{Tracker.TrackedReal{Float32},1}:
Which are obviously different but I do not know how to convert between them and cannot find anything online

Is there something I am doing wrong?

2

I was able to make this work by using

encoderA = Dense(10, 2)
encoderB(x) =  atan(Tracker.data(x[2]), Tracker.data(x[1]))
decoderA(x) = [cos(x), sin(x)]
decoderB = Dense(2, 10)

but I am also interested in back propagating to see what values are returned at the end of the encoding layers. Will this stop me from being able to do that?