# Zygote has a problem of derivation in the loss function

**URL:** <https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627>\
**Category:** New to Julia\
**Tags:** question, flux\
**Created:** [September 15, 2020, 8:53am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627 "2020-09-15T08:53:03Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![doli](https://avatars.discourse-cdn.com/v4/letter/d/ed655f/32.png) [@doli](https://discourse.julialang.org/u/doli)\
**Post date:** [September 15, 2020, 8:53am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/1 "2020-09-15T08:53:03Z")

</div>

I have a problem with Flux. Unfortunately Zygote cannot derive my loss function and I am not sure what it is. It would be nice if someone could help me with this problem.  
My loss function looks like this:

```julia
function loss_ungelabelt2(X,Y)
    
    a_list = Float32[]
    b_list = Float32[]
    mix_res = Float32[]
    
    λ::Float32 = 0.3
    loss::Float32 = 0
    
    for i in 1:2 #eigentlich size(X,4)  
        a = X[:,:,:,rand(1:size(X,4))] 
        b = X[:,:,:,rand(1:size(X,4))]
        
        mix = λ .* a .+ (1 - λ) .* b
        println(length(mix))
        println(size(mix))
        println("NEU")
        mix_res=reshape([mix_res...,mix...],size(mix)...,:) # einzelne Elemente 
        a_list=reshape([a_list...,a...],size(a)...,:)
        b_list=reshape([b_list...,b...],size(b)...,:)
        println(size(mix_res))
        println("neueRunde")
        
        
    end
    y_mix = model(mix_res)
    y_a = model(a_list)
    y_b = model(b_list)
    println("a", size(y_a))
    println("b", size(y_b))
    y_mix_ab = λ .* y_a .+ (1 - λ) .* y_b 
    println(length(y_mix_ab))
    
    quadrat_loss = sum(abs2,y_mix .- y_mix_ab)
    
    return quadrat_loss
    
end

```

I get this error:

```julia
MethodError: no method matching size(::NTuple{784,Float32})
Closest candidates are:
  size(::Tuple, !Matched::Integer) at tuple.jl:22
  size(!Matched::Flux.OneHotVector) at /Users/lisa/.julia/packages/Flux/IjMZL/src/onehot.jl:8
  size(!Matched::ZMQ.Message) at /Users/lisa/.julia/packages/ZMQ/R3wSD/src/message.jl:95
  ...

Stacktrace:
 [1] unbroadcast(::Array{Float32,3}, ::NTuple{784,Float32}) at /Users/lisa/.julia/packages/Zygote/seGHk/src/lib/broadcast.jl:53
 [2] (::Zygote.var"#1107#1109"{NTuple{784,Float32}})(::Array{Float32,3}) at /Users/lisa/.julia/packages/Zygote/seGHk/src/lib/broadcast.jl:74
 [3] map(::Zygote.var"#1107#1109"{NTuple{784,Float32}}, ::Tuple{Array{Float32,3},Array{Float32,3}}) at ./tuple.jl:158
 [4] (::Zygote.var"#1106#1108"{Tuple{Array{Float32,3},Array{Float32,3}}})(::NTuple{784,Float32}) at /Users/lisa/.julia/packages/Zygote/seGHk/src/lib/broadcast.jl:74
 [5] (::Zygote.var"#3852#back#1110"{Zygote.var"#1106#1108"{Tuple{Array{Float32,3},Array{Float32,3}}}})(::NTuple{784,Float32}) at /Users/lisa/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [6] loss_ungelabelt2 at ./In[13]:14 [inlined]
 [7] (::typeof(∂(loss_ungelabelt2)))(::Float32) at /Users/lisa/.julia/packages/Zygote/seGHk/src/compiler/interface2.jl:0
 [8] (::Zygote.var"#41#42"{typeof(∂(loss_ungelabelt2))})(::Float32) at /Users/lisa/.julia/packages/Zygote/seGHk/src/compiler/interface.jl:45
 [9] gradient(::Function, ::Array{Float32,4}, ::Vararg{Any,N} where N) at /Users/lisa/.julia/packages/Zygote/seGHk/src/compiler/interface.jl:54
 [10] top-level scope at In[14]:1
 [11] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091

```

I would be very grateful for an answer.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 15, 2020, 9:04am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/2 "2020-09-15T09:04:05Z")

</div>

> [@doli](#):
>
> `size(::NTuple{784,Float32})`

what abt this `size(::NTuple{N,T}) where {N, T} = N`

---

<div class="post-metadata">

**Author:** ![doli](https://avatars.discourse-cdn.com/v4/letter/d/ed655f/32.png) [@doli](https://discourse.julialang.org/u/doli)\
**Post date:** [September 15, 2020, 9:14am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/3 "2020-09-15T09:14:20Z")

</div>

What exactly do you mean by this?  
Because I am not quite sure where this mistake comes from. The sizes in the code are multidimensional. And only the lengths are 784.

---

<div class="post-metadata">

**Author:** ![zgornel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zgornel/32/217487_2.png) [@zgornel](https://discourse.julialang.org/u/zgornel)\
**Post date:** [September 15, 2020, 9:32am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/4 "2020-09-15T09:32:48Z")

</div>

the `size` function does not support arguments of type `NTuple{N,Float32}`. @xiaodai 's solution is to add a method to size as: `size(::NTuple{N,T}) where {N, T} = N`. It should solve the problem reported i.e. missing a `size` method.

---

<div class="post-metadata">

**Author:** ![doli](https://avatars.discourse-cdn.com/v4/letter/d/ed655f/32.png) [@doli](https://discourse.julialang.org/u/doli)\
**Post date:** [September 15, 2020, 9:47am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/5 "2020-09-15T09:47:59Z")

</div>

> [@zgornel](#):
>
> the `size` function does not support arguments of type `NTuple{N,Float32}` . @xiaodai 's solution is to add a method to size as: `size(::NTuple{N,T}) where {N, T} = N` . It should solve the error reported.

Ah okay. Thanks a lot!  
Sry, I don’t usually work with Julia and I am a little helpless.  
But if I add the line `Base.size(::NTuple{N,T}) where {N, T} = N` I get an `argumentError: invalid tuple dimension 2`

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [September 15, 2020, 10:19am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/6 "2020-09-15T10:19:43Z")

</div>

You could do that, it might fix it. (Or it might not)  
Its not the real solution though, and it is kind of a bad thing to do, it is type piracy (overloading a function you don’t own on a set of types none of which you own).

This is a bug in Zygote. you should report it on the [Zygote issue tracker](https://github.com/FluxML/Zygote.jl/issues/).  
Its not something you have done wrong.

I did some poking  
The error is this line [https://github.com/FluxML/Zygote.jl/blob/ed366f32c0f520567526040d9f8acaf0d83613c3/src/lib/broadcast.jl#L52](https://github.com/FluxML/Zygote.jl/blob/ed366f32c0f520567526040d9f8acaf0d83613c3/src/lib/broadcast.jl#L52)  
where zygote tries to take `size(x̄)`  
`x̄` is turning out to be a `Tuple` which don’t have `size` defined on them.  
Only `length`.  
However, that is just a symptom, as that line should be fine,  
because if `x` is an `Array`,  
then `x̄` should also be an `Array`.

So something is going wrong somewhere else that is causing it to be a `Tuple`.  
It needs to be fixed in Zygote.

---

<div class="post-metadata">

**Author:** ![doli](https://avatars.discourse-cdn.com/v4/letter/d/ed655f/32.png) [@doli](https://discourse.julialang.org/u/doli)\
**Post date:** [September 15, 2020, 10:43am UTC](https://discourse.julialang.org/t/zygote-has-a-problem-of-derivation-in-the-loss-function/46627/7 "2020-09-15T10:43:02Z")

</div>

Thank you for the effort! I have already reported it and I hope it can be fixed without much effort and time.