# Training on Simple Loss gives error: No method matching +(::Float64, ::Array{Float64, 2})

**URL:** <https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047>\
**Category:** Machine Learning\
**Created:** [March 26, 2021, 8:19pm UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047 "2021-03-26T20:19:32Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![kimlaberinto](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kimlaberinto/32/21534_2.png) [@kimlaberinto](https://discourse.julialang.org/u/kimlaberinto)\
**Post date:** [March 26, 2021, 8:19pm UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/1 "2021-03-26T20:19:32Z")

</div>

Hello everyone!

I’m getting a weird error with Flux/DiffEqFlux with a simple loss function when I try to train. I’m not sure why it’s giving me a `ERROR: LoadError: MethodError: no method matching +(::Float64, ::Array{Float64,2}` . Loss function works when I evaluate it myself.

Could anyone help me with suggestions for how to debug or better code it?

(It’ll be part of a slightly bigger SciML problem, which is why I’d like to use DiffEqFlux)

Simplified code and runnable example below to see error.

```julia
using DiffEqFlux
using Flux

U = FastChain(FastDense(1, 10, sigmoid), FastDense(10, 1))
function loss(θ)
    # Evaluate U neural network at y and return scalar
    nn_windspeed = y -> U(y, θ)[1]
    y = -5:0.1:5
    windspeeds = nn_windspeed.(y)
    truewinds = y.^2 #example
    return sum(abs2, (windspeeds .- truewinds))
end
loss(initial_params(U)) #works
DiffEqFlux.sciml_train(loss, initial_params(U), ADAM(1.0e-2), maxiters = 200) #error

```

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [March 26, 2021, 9:15pm UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/2 "2021-03-26T21:15:44Z")

</div>

Try `y = Float32.(-5:0.1:5)`, I believe that that is the source of the problem-

---

<div class="post-metadata">

**Author:** ![kimlaberinto](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kimlaberinto/32/21534_2.png) [@kimlaberinto](https://discourse.julialang.org/u/kimlaberinto)\
**Post date:** [March 27, 2021, 12:53am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/3 "2021-03-27T00:53:40Z")

</div>

Thank you. Unfortunately `y = Float32.(-5:0.1:5)` didn’t work. I still get the same error.

There seems to be a problem with `nn_windspeed = y -> U(y, θ)[1]` and `windspeeds = nn_windspeed.(y)`. I suspect that because of how that line gets highlighted in the stacktrace `[10]` below.

```julia
ERROR: LoadError: MethodError: no method matching +(::Float64, ::Array{Float64,2})
For element-wise addition, use broadcasting with dot syntax: scalar .+ array
Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...) at operators.jl:538
  +(::ChainRulesCore.DoesNotExist, ::Any) at ~/.julia/packages/ChainRulesCore/DBATp/src/differential_arithmetic.jl:23
  +(::ChainRulesCore.One, ::Any) at ~/.julia/packages/ChainRulesCore/DBATp/src/differential_arithmetic.jl:94
  ...
Stacktrace:
 [1] accum(::Float64, ::Array{Float64,2}) at ~/.julia/packages/Zygote/pM10l/src/lib/lib.jl:8
 [2] _broadcast_getindex_evalf at ./broadcast.jl:648 [inlined]
 [3] _broadcast_getindex at ./broadcast.jl:621 [inlined]
 [4] getindex at ./broadcast.jl:575 [inlined]
 [5] copy at ./broadcast.jl:876 [inlined]
 [6] materialize at ./broadcast.jl:837 [inlined]
 [7] accum(::Array{Float64,1}, ::Array{Array{Float64,2},1}) at ~/.julia/packages/Zygote/pM10l/src/lib/lib.jl:16
 [8] getindex at ./tuple.jl:24 [inlined]
 [9] gradindex at ~/.julia/packages/Zygote/pM10l/src/compiler/reverse.jl:12 [inlined]
 [10] loss at current_file_script.jl:9 [inlined]
 [11] (::typeof(∂(loss)))(::Float64) at ~/.julia/packages/Zygote/pM10l/src/compiler/interface2.jl:0
 [12] #69 at ~/.julia/packages/DiffEqFlux/lS4Sa/src/train.jl:3 [inlined]
 [13] #178 at ~/.julia/packages/Zygote/pM10l/src/lib/lib.jl:194 [inlined]
 [14] #1698#back at ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59 [inlined]
 [15] OptimizationFunction at ~/.julia/packages/SciMLBase/fypD8/src/problems/basic_problems.jl:107 [inlined]
 [16] #178 at ~/.julia/packages/Zygote/pM10l/src/lib/lib.jl:194 [inlined]
 [17] #1698#back at ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59 [inlined]
 [18] OptimizationFunction at ~/.julia/packages/SciMLBase/fypD8/src/problems/basic_problems.jl:107 [inlined]
 [19] #178 at ~/.julia/packages/Zygote/pM10l/src/lib/lib.jl:194 [inlined]
 [20] (::Zygote.var"#1698#back#180"{Zygote.var"#178#179"{typeof(∂(λ)),Tuple{Tuple{Nothing,Nothing},Int64}}})(::Float64) at ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [21] #8 at ~/.julia/packages/GalacticOptim/Ha1cY/src/solve.jl:94 [inlined]
 [22] (::typeof(∂(λ)))(::Float64) at ~/.julia/packages/Zygote/pM10l/src/compiler/interface2.jl:0
 [23] (::Zygote.var"#69#70"{Zygote.Params,Zygote.Context,typeof(∂(λ))})(::Float64) at ~/.julia/packages/Zygote/pM10l/src/compiler/interface.jl:252
 [24] gradient(::Function, ::Zygote.Params) at ~/.julia/packages/Zygote/pM10l/src/compiler/interface.jl:59
 [25] __solve(::SciMLBase.OptimizationProblem{false,SciMLBase.OptimizationFunction{false,GalacticOptim.AutoZygote,SciMLBase.OptimizationFunction{true,GalacticOptim.AutoZygote,DiffEqFlux.var"#69#70"{typeof(loss)},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},GalacticOptim.var"#146#156"{GalacticOptim.var"#145#155"{SciMLBase.OptimizationFunction{true,GalacticOptim.AutoZygote,DiffEqFlux.var"#69#70"{typeof(loss)},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing}},GalacticOptim.var"#149#159"{GalacticOptim.var"#145#155"{SciMLBase.OptimizationFunction{true,GalacticOptim.AutoZygote,DiffEqFlux.var"#69#70"{typeof(loss)},Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Nothing}},GalacticOptim.var"#154#164",Nothing,Nothing,Nothing},Array{Float32,1},SciMLBase.NullParameters,Nothing,Nothing,Nothing,Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol},NamedTuple{(:maxiters,),Tuple{Int64}}}}, ::ADAM, ::Base.Iterators.Cycle{Tuple{GalacticOptim.NullData}}; maxiters::Int64, cb::Function, progress::Bool, save_best::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ~/.julia/packages/GalacticOptim/Ha1cY/src/solve.jl:93
 [26] #solve#450 at ~/.julia/packages/SciMLBase/fypD8/src/solve.jl:3 [inlined]
 [27] sciml_train(::typeof(loss), ::Array{Float32,1}, ::ADAM, ::GalacticOptim.AutoZygote; lower_bounds::Nothing, upper_bounds::Nothing, kwargs::Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol},NamedTuple{(:maxiters,),Tuple{Int64}}}) at ~/.julia/packages/DiffEqFlux/lS4Sa/src/train.jl:6
 [28] top-level scope at current_file_script.jl:14
 [29] include(::String) at ./client.jl:457
 [30] top-level scope at REPL[20]:1
 [31] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.3/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
in expression starting at current_file_script.jl:14

```

---

<div class="post-metadata">

**Author:** ![arlk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arlk/32/2465_2.png) [@arlk](https://discourse.julialang.org/u/arlk)\
**Post date:** [March 27, 2021, 1:16am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/4 "2021-03-27T01:16:21Z")

</div>

`y` needs to be a row vector and you don’t have to broadcast it

```julia
function loss(θ)
    y = (-5:0.1:5)'
    windspeeds = U(y, θ)
    truewinds = y.^2 #example
    return sum(abs2, (windspeeds .- truewinds))
end

```

---

<div class="post-metadata">

**Author:** ![kimlaberinto](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kimlaberinto/32/21534_2.png) [@kimlaberinto](https://discourse.julialang.org/u/kimlaberinto)\
**Post date:** [March 27, 2021, 1:33am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/5 "2021-03-27T01:33:26Z")

</div>

Thank you so much! This was the solution and didn’t have to do the weird broadcast and indexing. Definitely going to be much more careful with when the input should be a row vector or not and how its arranged.

---

<div class="post-metadata">

**Author:** ![arlk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arlk/32/2465_2.png) [@arlk](https://discourse.julialang.org/u/arlk)\
**Post date:** [March 27, 2021, 1:35am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/6 "2021-03-27T01:35:22Z")

</div>

Just fyi broadcasting might help with performance for very large batch sizes. For that you can do something like `windspeeds = U.(eachcol(y), Ref(θ))`

---

<div class="post-metadata">

**Author:** ![kimlaberinto](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kimlaberinto/32/21534_2.png) [@kimlaberinto](https://discourse.julialang.org/u/kimlaberinto)\
**Post date:** [March 27, 2021, 1:38am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/7 "2021-03-27T01:38:13Z")

</div>

Thank you so much for the tip! I didn’t know about eachcol and being able to use broadcast in that way.

Do you know why it gives a performance boost for large datasets?

---

<div class="post-metadata">

**Author:** ![arlk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arlk/32/2465_2.png) [@arlk](https://discourse.julialang.org/u/arlk)\
**Post date:** [March 27, 2021, 1:43am UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/8 "2021-03-27T01:43:35Z")

</div>

I haven’t used the fast layers in DiffEqFlux much, but IIRC they are optimized to run fast if the size of the layers is \< 100-200 - the same would hold for the batch of data you are feeding in as well. If your batches are in the 1000s then I’m not sure how much benefit you get from using these fast layers. But it’s possible I’m wrong about this, I just use Flux and for that it is definitely faster to pass the dataset as a matrix.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 27, 2021, 12:35pm UTC](https://discourse.julialang.org/t/training-on-simple-loss-gives-error-no-method-matching-float64-array-float64-2/58047/9 "2021-03-27T12:35:20Z")

</div>

That is true.