# How to specifying coefficient in NNlib.leakyrelu?

**URL:** https://discourse.julialang.org/t/how-to-specifying-coefficient-in-nnlib-leakyrelu/31009
**Category:** Machine Learning
**Tags:** flux
**Created:** [November 12, 2019, 4:15pm UTC](https://discourse.julialang.org/t/how-to-specifying-coefficient-in-nnlib-leakyrelu/31009 "2019-11-12T16:15:15Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![taot](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/taot/32/11257_2.png) [@taot](https://discourse.julialang.org/u/taot)
#### Post date: [November 12, 2019, 4:15pm UTC](https://discourse.julialang.org/t/how-to-specifying-coefficient-in-nnlib-leakyrelu/31009/1 "2019-11-12T16:15:15Z")

</div>

Hi,

I am trying to specify the coefficient in leayrelu activation function, something like this:

```julia
conv1 = Conv( (4,4), 3=>16, x -> leakyrelu(x, 0.02), stride=2, pad=(1,1) )
conv2 = Conv( (4,4), 16=>32, x -> leakyrelu(x, 0.02), stride=2, pad=(1,1) )

data = randn(Float32, 64, 64, 3, 8);
conv2(conv1(data))

```

And I got this error:

```julia
UndefVarError: project not defined

Stacktrace:
 [1] macro expansion at /home/taot/.julia/packages/NNlib/mxWRT/src/impl/conv_direct.jl:76 [inlined]
 [2] #conv_direct!#137(::Float64, ::Bool, ::typeof(NNlib.conv_direct!), ::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:211
 [3] conv_direct! at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:180 [inlined]
 [4] #conv!#79(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv!), ::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:97
 [5] conv!(::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:95
 [6] #conv!#44(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv!), ::Array{Float64,4}, ::Array{Float64,4}, ::Array{Float32,4}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:68
 [7] conv! at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:68 [inlined]
 [8] macro expansion at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:114 [inlined]
 [9] #conv#85(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv), ::Array{Float64,4}, ::Array{Float32,4}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:211
 [10] #_forward#524 at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:180 [inlined]
 [11] _forward(::typeof(conv), ::TrackedArray{…,Array{Float64,4}}, ::TrackedArray{…,Array{Float32,4}}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at ./none:0
 [12] #track#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Tracker.track), ::typeof(conv), ::TrackedArray{…,Array{Float64,4}}, ::Vararg{Any,N} where N) at /home/taot/.julia/packages/Tracker/SAr25/src/Tracker.jl:51
 [13] track at /home/taot/.julia/packages/Tracker/SAr25/src/Tracker.jl:51 [inlined]
 [14] #conv#521 at /home/taot/.julia/packages/Tracker/SAr25/src/lib/array.jl:443 [inlined]
 [15] conv at /home/taot/.julia/packages/Tracker/SAr25/src/lib/array.jl:443 [inlined]
 [16] (::Conv{2,2,getfield(Main, Symbol("##10#11")),TrackedArray{…,Array{Float32,4}},TrackedArray{…,Array{Float32,1}}})(::TrackedArray{…,Array{Float64,4}}) at /home/taot/.julia/packages/Flux/dkJUV/src/layers/conv.jl:55
 [17] top-level scope at In[30]:1

```

I also tried not to specify the coefficient parameter, and it works fine:

```julia
conv1 = Conv( (4,4), 3=>16, leakyrelu, stride=2, pad=(1,1) )
conv2 = Conv( (4,4), 16=>32, leakyrelu, stride=2, pad=(1,1) )

```

I looked at the source code of leakyrelu which seems to me just a normal function. I am new to Julia and Flux, I’m wondering what’s the difference between leakyrelu and the activation function defined by “x → leakyrelu(x, 0.02)” ?

And how should I specify the coefficient in this case?

Thanks!

---

<div class="post-metadata">

### Author: ![findmyway](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/findmyway/32/4946_2.png) [@findmyway](https://discourse.julialang.org/u/findmyway)
#### Post date: [November 15, 2019, 7:33am UTC](https://discourse.julialang.org/t/how-to-specifying-coefficient-in-nnlib-leakyrelu/31009/2 "2019-11-15T07:33:43Z")

</div>

I can confirm that your code works on the master branch and the latest stable version (v0.9.0).

Try update?

---

<div class="post-metadata">

### Author: ![taot](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/taot/32/11257_2.png) [@taot](https://discourse.julialang.org/u/taot)
#### Post date: [November 15, 2019, 9:28am UTC](https://discourse.julialang.org/t/how-to-specifying-coefficient-in-nnlib-leakyrelu/31009/3 "2019-11-15T09:28:06Z")

</div>

Thanks for your replay, findmyway!

I checked my Flux version, and it was indeed v0.9.0.

Then I tried to remove Flux and install it again, and the problem goes away. That’s weird, maybe something got polluted in my old installation which I am not able to find out.