Leaky relu with Flux

Specific Domains Machine Learning
1

Any ideas how to implement LeakyRelu, getting a no method matching error on isless. Also I had to implement my own std function:

Thanks!

mystd(x) = sqrt(sum((x-mean(x)).^2)/(length(x)-1))
lrelu(x) = isless(x,0.0) ? 0.3*x : x
function nlrelu(x)
return lrelu(mystd.(x.-mean(x)))
end
model = Chain(
Dense(2,128),
nlrelu,
Dense(128,128),
nlrelu,
Dense(128,1)
)
model(randn(2))

the error:

MethodError: no method matching isless(::TrackedArray{…,Array{Float64,1}}, ::Float64)WARNING: Error showing method candidates, aborted

Stacktrace:
[1] lrelu at ./In[7]:3 [inlined]
[2] nlrelu(::TrackedArray{…,Array{Float64,1}}) at ./In[7]:6
[3] mapfoldl_impl(::Base.#identity, ::Flux.##45#46, ::Array{Float64,1}, ::Array{Any,1}, ::Int64) at ./reduce.jl:46
[4] (::Flux.Chain)(::Array{Float64,1}) at /home/ariel/.julia/v0.6/Flux/src/layers/basic.jl:28
[5] include_string(::String, ::String) at ./loading.jl:515

2

actually, I solved it like this:

mystd(x) = sqrt.(sum((x.-mean(x)).^2) ./ (length(x)-1))
lrelu(x) = max.(0.3*x, x)

3 Likes