# Custom Flux loss function \`culiteral\_pow()\` error

**URL:** <https://discourse.julialang.org/t/custom-flux-loss-function-culiteral-pow-error/27671>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [August 18, 2019, 5:27am UTC](https://discourse.julialang.org/t/custom-flux-loss-function-culiteral-pow-error/27671 "2019-08-18T05:27:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jchen975](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jchen975/32/8994_2.png) [@jchen975](https://discourse.julialang.org/u/jchen975)\
**Post date:** [August 18, 2019, 5:27am UTC](https://discourse.julialang.org/t/custom-flux-loss-function-culiteral-pow-error/27671/1 "2019-08-18T05:27:07Z")

</div>

Hi, I’m trying to build a MLP for a regression problem. A sample of my data looks like this:  
 x= \left[{\begin{array}{c} \hat{m\_1}\\ \vdots \\ \hat{m\_k}\\ \hat{n\_1}\\ \vdots \\ \hat{n\_k}\\ P \\ Q \\ \end{array} } \right] , where P, Q similarly goes from 1 to k.  
A sample of my target looks like this:  
 y= \left[{\begin{array}{c} m\_1 \\ \vdots \\ m\_k \\ n\_1 \\ \vdots \\ n\_k \\ \end{array} } \right] .

For this particular problem, \hat{m\_i} and m\_i are very close before any training, but not \hat{n\_i} and n\_i. There is another model where I’m passing in the final test set prediction, and model(\hat{n\_i}) needs to be as close to n\_i as possible, and I don’t care so much for \hat{m\_i} and m\_i (obviously, I want them to be close, but it’s not as important). So, I tried modifying the plain MSE loss into this:

```julia
loss = Flux.mse(model(x)[1:k, :], y[1:k, :]) + λ*Flux.mse(model(x)[k+1:end, :], y[k+1:end, :])

```

, where \lambda is greater than 1 and subject to tuning.

However, I’m getting the following error:

```julia
ERROR: MethodError: no method matching culiteral_pow(::typeof(^), ::ForwardDiff.Dual{Nothing,Float32,1}, ::Val{2})
Closest candidates are:
  culiteral_pow(::typeof(^), ::Union{Float32, Float64}, ::Val{2}) at C:\Users\me\.julia\packages\CuArrays\eFBar\src\broadcast.jl:46
  culiteral_pow(::typeof(^), ::Union{Float32, Float64}, ::Val{p}) where p at C:\Users\me\.julia\packages\CuArrays\eFBar\src\broadcast.jl:48
  culiteral_pow(::typeof(^), ::Union{Float32, Float64}, ::Val{0}) at C:\Users\me\.julia\packages\CuArrays\eFBar\src\broadcast.jl:44
  ...
Stacktrace:
 [1] (::getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##1#3"))},getfieldBase.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcat, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CuArrays.culiteral_pow)})(::Function, ::ForwardDiffDual{Nothing,Float32,1}, ::ForwardDiff.Dual{Nothing,Float32,1}, ::Val{2}) at .\broadcast.jl:298
 [2] partial(::getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##1#3"))},gtfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.roadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CuArrays.culiteral_pow)}, ::Float64, ::Int64 ::Function, ::Float32, ::Float32, ::Val{2}) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\array.jl:480
 [3] _broadcast_getindex_evalf at .\broadcast.jl:578 [inlined]
 [4] _broadcast_getindex at .\broadcast.jl:551 [inlined]
 [5] getindex at .\broadcast.jl:511 [inlined]
 [6] copy at .\broadcast.jl:787 [inlined]
 [7] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{CuArray},Nothing,typeof(Tracker.partial),Tuple{Base.RefValue{getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Bae.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##5#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CuArrays.culiteral_pow)}},CuArray{Float64,2},Int64,Base.RefValue{typeof(^)},CuArray{Float32,2},CuArray{Float32,2},Base.RefValue{Val{2}}}}) at .\broadcast.j:753
 [8] broadcast(::typeof(Tracker.partial), ::Base.RefValue{getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){gtfield(Base.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Boadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CArrays.culiteral_pow)}}, ::CuArray{Float64,2}, ::Int64, ::Vararg{Any,N} where N) at .\broadcast.jl:707
 [9] ∇broadcast(::typeof(Tracker.partial), ::Base.RefValue{getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){etfield(Base.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.roadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(uArrays.culiteral_pow)}}, ::CuArray{Float64,2}, ::Int64, ::Base.RefValue{typeof(^)}, ::TrackedArray{…,CuArray{Float32,2}}, ::CuArray{Float32,2}, ::Base.RefValue{Val{2}}) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\rray.jl:484
 [10] copy(::Base.Broadcast.Broadcasted{Tracker.TrackedStyle,Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},typeof(Tracker.partial),Tuple{Base.RefValue{getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6)){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(BaseBroadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcst, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CuArrays.culiteral_pow)}},CuArray{Float64,2},Int64,Base.RefValue{typeof(^)},TrackedArray{…,CuArray{Float32,2}},CuArray{Float32,2},Base.RefVaue{Val{2}}}}) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\array.jl:515
 [11] materialize at .\broadcast.jl:753 [inlined]
 [12] #547 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\array.jl:488 [inlined]
 [13] macro expansion at .\sysimg.jl:300 [inlined]
 [14] ntuple at .\sysimg.jl:296 [inlined]
 [15] (::getfield(Tracker, Symbol("#back#548")){4,getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(ase.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)}},typeof(CuArrays.uliteral_pow)},Tuple{Base.RefValue{typeof(^)},TrackedArray{…,CuArray{Float32,2}},CuArray{Float32,2},Base.RefValue{Val{2}}}})(::CuArray{Float64,2}) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\array.jl:488
 [16] back_(::Tracker.Call{getfield(Tracker, Symbol("#back#548")){4,getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("#5#6")){getfield(Base.Broadcast, Symbol("##1#3"))},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfild(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(-)},typeof(CuArrays.culiteral_pow)},Tuple{Base.RefValue{typeof(^)},TrackedArray{…,CuArray{Float32,2}},CuArray{Float32,2},Base.RefValue{Val{2}}}},Tuple{Nothing,Tracker.Tracked{CuArray{Float32,2}},Nothing,Nothing}}, ::CuArray{Float642}, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:35
 [17] back(::Tracker.Tracked{CuArray{Float32,2}}, ::CuArray{Float64,2}, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:58
 [18] #13 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38 [inlined]
 [19] foreach at .\abstractarray.jl:1867 [inlined]
 [20] back_(::Tracker.Call{getfield(Tracker, Symbol("##482#483")){TrackedArray{…,CuArray{Float32,2}}},Tuple{Tracker.Tracked{CuArray{Float32,2}}}}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:8
 [21] back(::Tracker.Tracked{Float32}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:58
 [22] #13 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38 [inlined]
 [23] foreach at .\abstractarray.jl:1867 [inlined]
 [24] back_(::Tracker.Call{getfield(Tracker, Symbol("##278#281")){Rational{Int64}},Tuple{Tracker.Tracked{Float32},Nothing}}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38
 [25] back(::Tracker.Tracked{Float32}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:58
 [26] #13 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38 [inlined]
 [27] foreach at .\abstractarray.jl:1867 [inlined]
 [28] back_(::Tracker.Call{getfield(Tracker, Symbol("##279#282")){Float64},Tuple{Nothing,Tracker.Tracked{Float32}}}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38
 [29] back(::Tracker.Tracked{Float64}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:58
 [30] #13 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38 [inlined]
 [31] foreach at .\abstractarray.jl:1867 [inlined]
 [32] back_(::Tracker.Call{getfield(Tracker, Symbol("##253#256")),Tuple{Tracker.Tracked{Float64},Tracker.Tracked{Float32}}}, ::Float64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:38
 [33] back(::Tracker.Tracked{Float64}, ::Int64, ::Bool) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:58
 [34] #back!#15 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:77 [inlined]
 [35] #back! at .\none:0 [inlined]
 [36] #back!#32 at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\real.jl:16 [inlined]
 [37] back!(::Tracker.TrackedReal{Float64}) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\lib\real.jl:14
 [38] gradient_(::getfield(Flux.Optimise, Symbol("##14#20")){getfield(Main, Symbol("#loss#23")){Float64},Tuple{CuArray{Float32,2},CuArray{Float32,2}}}, ::Tracker.Params) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\backjl:4
 [39] #gradient#24(::Bool, ::Function, ::Function, ::Tracker.Params) at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:164
 [40] gradient at C:\Users\me\.julia\packages\Tracker\RRYy6\src\back.jl:164 [inlined]
 [41] macro expansion at C:\Users\me\.julia\packages\Flux\qXNjB\src\optimise\train.jl:71 [inlined]
 [42] macro expansion at C:\Users\me\.julia\packages\Juno\Drrdg\src\progress.jl:119 [inlined]
 [43] #train!#12(::getfield(Flux.Optimise, Symbol("##16#22")), ::Function, ::Function, ::Tracker.Params, ::Array{Tuple{CuArray{Float32,2},CuArray{Float32,2}},1}, ::ADAM) at C:\Users\me\.julia\packages\Flux\qXNjB\src\optimise\train.jl:69
 [44] train!(::Function, ::Tracker.Params, ::Array{Tuple{CuArray{Float32,2},CuArray{Float32,2}},1}, ::ADAM) at C:\Users\me\.julia\packages\Flux\qXNjB\src\optimise\train.jl:67
 [45] train_net(::String, ::Array{Float32,2}, ::Array{Float32,2}, ::Float64, ::Int64, ::Int64, ::Float64, ::Int64, ::Int64, ::Bool) at F:\work\Merge\train_nc.jl:127
 [46] train_net(::String, ::Array{Float32,2}, ::Array{Float32,2}, ::Float64, ::Int64, ::Int64) at F:\work\Merge\train_nc.jl:52
 [47] main(::Array{String,1}) at .\util.jl:156
 [48] top-level scope at none:0

```

I’d appreciate any suggestions on how to solve this, and/or a better custom loss function.

---

<div class="post-metadata">

**Author:** ![WschW](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wschw/32/6575_2.png) [@WschW](https://discourse.julialang.org/u/WschW)\
**Post date:** [August 18, 2019, 5:42am UTC](https://discourse.julialang.org/t/custom-flux-loss-function-culiteral-pow-error/27671/2 "2019-08-18T05:42:36Z")

</div>

Found a corresponding issue on GitHub:

[https://github.com/JuliaGPU/CuArrays.jl/issues/378](https://github.com/JuliaGPU/CuArrays.jl/issues/378)

Looks like there is a fix but it hasn’t been merged yet.

[https://github.com/JuliaGPU/CuArrays.jl/pull/380](https://github.com/JuliaGPU/CuArrays.jl/pull/380)

---

<div class="post-metadata">

**Author:** ![jchen975](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jchen975/32/8994_2.png) [@jchen975](https://discourse.julialang.org/u/jchen975)\
**Post date:** [August 18, 2019, 6:16am UTC](https://discourse.julialang.org/t/custom-flux-loss-function-culiteral-pow-error/27671/3 "2019-08-18T06:16:27Z")

</div>

I actually saw these earlier too, but thanks!

Do you know if my new loss definition is correct though? I’m not sure if I can use regular array indexing on that.