# Leaky relu with Flux

**URL:** <https://discourse.julialang.org/t/leaky-relu-with-flux/6377>\
**Category:** Machine Learning\
**Created:** [October 10, 2017, 1:02pm UTC](https://discourse.julialang.org/t/leaky-relu-with-flux/6377 "2017-10-10T13:02:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![skariel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skariel/32/2139_2.png) [@skariel](https://discourse.julialang.org/u/skariel)\
**Post date:** [October 10, 2017, 1:02pm UTC](https://discourse.julialang.org/t/leaky-relu-with-flux/6377/1 "2017-10-10T13:02:30Z")

</div>

Any ideas how to implement LeakyRelu, getting a no method matching error on isless. Also I had to implement my own std function:

Thanks!

mystd(x) = sqrt(sum((x-mean(x)).^2)/(length(x)-1))  
lrelu(x) = isless(x,0.0) ? 0.3\*x : x  
function nlrelu(x)  
return lrelu(mystd.(x.-mean(x)))  
end  
model = Chain(  
Dense(2,128),  
nlrelu,  
Dense(128,128),  
nlrelu,  
Dense(128,1)  
)  
model(randn(2))

the error:

MethodError: no method matching isless(::TrackedArray{…,Array{Float64,1}}, ::Float64)WARNING: Error showing method candidates, aborted

Stacktrace:  
[1] lrelu at ./In[7]:3 [inlined]  
[2] nlrelu(::TrackedArray{…,Array{Float64,1}}) at ./In[7]:6  
[3] mapfoldl\_impl(::Base.#identity, ::Flux.##45#46, ::Array{Float64,1}, ::Array{Any,1}, ::Int64) at ./reduce.jl:46  
[4] (::Flux.Chain)(::Array{Float64,1}) at /home/ariel/.julia/v0.6/Flux/src/layers/basic.jl:28  
[5] include\_string(::String, ::String) at ./loading.jl:515

---

<div class="post-metadata">

**Author:** ![skariel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skariel/32/2139_2.png) [@skariel](https://discourse.julialang.org/u/skariel)\
**Post date:** [October 10, 2017, 1:09pm UTC](https://discourse.julialang.org/t/leaky-relu-with-flux/6377/2 "2017-10-10T13:09:03Z")

</div>

actually, I solved it like this:

mystd(x) = sqrt.(sum((x.-mean(x)).^2) ./ (length(x)-1))  
lrelu(x) = max.(0.3\*x, x)