How to specifying coefficient in NNlib.leakyrelu?

Hi,

I am trying to specify the coefficient in leayrelu activation function, something like this:

conv1 = Conv( (4,4), 3=>16, x -> leakyrelu(x, 0.02), stride=2, pad=(1,1) )
conv2 = Conv( (4,4), 16=>32, x -> leakyrelu(x, 0.02), stride=2, pad=(1,1) )

data = randn(Float32, 64, 64, 3, 8);
conv2(conv1(data))

And I got this error:

UndefVarError: project not defined

Stacktrace:
 [1] macro expansion at /home/taot/.julia/packages/NNlib/mxWRT/src/impl/conv_direct.jl:76 [inlined]
 [2] #conv_direct!#137(::Float64, ::Bool, ::typeof(NNlib.conv_direct!), ::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:211
 [3] conv_direct! at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:180 [inlined]
 [4] #conv!#79(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv!), ::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:97
 [5] conv!(::Array{Float64,5}, ::Array{Float64,5}, ::Array{Float32,5}, ::DenseConvDims{3,(4, 4, 1),16,32,(2, 2, 1),(1, 1, 1, 1, 0, 0),(1, 1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:95
 [6] #conv!#44(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv!), ::Array{Float64,4}, ::Array{Float64,4}, ::Array{Float32,4}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:68
 [7] conv! at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:68 [inlined]
 [8] macro expansion at /home/taot/.julia/packages/NNlib/mxWRT/src/conv.jl:114 [inlined]
 [9] #conv#85(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(conv), ::Array{Float64,4}, ::Array{Float32,4}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:211
 [10] #_forward#524 at /home/taot/.julia/packages/TimerOutputs/ohPOH/src/TimerOutput.jl:180 [inlined]
 [11] _forward(::typeof(conv), ::TrackedArray{…,Array{Float64,4}}, ::TrackedArray{…,Array{Float32,4}}, ::DenseConvDims{2,(4, 4),16,32,(2, 2),(1, 1, 1, 1),(1, 1),false}) at ./none:0
 [12] #track#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Tracker.track), ::typeof(conv), ::TrackedArray{…,Array{Float64,4}}, ::Vararg{Any,N} where N) at /home/taot/.julia/packages/Tracker/SAr25/src/Tracker.jl:51
 [13] track at /home/taot/.julia/packages/Tracker/SAr25/src/Tracker.jl:51 [inlined]
 [14] #conv#521 at /home/taot/.julia/packages/Tracker/SAr25/src/lib/array.jl:443 [inlined]
 [15] conv at /home/taot/.julia/packages/Tracker/SAr25/src/lib/array.jl:443 [inlined]
 [16] (::Conv{2,2,getfield(Main, Symbol("##10#11")),TrackedArray{…,Array{Float32,4}},TrackedArray{…,Array{Float32,1}}})(::TrackedArray{…,Array{Float64,4}}) at /home/taot/.julia/packages/Flux/dkJUV/src/layers/conv.jl:55
 [17] top-level scope at In[30]:1

I also tried not to specify the coefficient parameter, and it works fine:

conv1 = Conv( (4,4), 3=>16, leakyrelu, stride=2, pad=(1,1) )
conv2 = Conv( (4,4), 16=>32, leakyrelu, stride=2, pad=(1,1) )

I looked at the source code of leakyrelu which seems to me just a normal function. I am new to Julia and Flux, I’m wondering what’s the difference between leakyrelu and the activation function defined by “x -> leakyrelu(x, 0.02)” ?

And how should I specify the coefficient in this case?

Thanks!

I can confirm that your code works on the master branch and the latest stable version (v0.9.0).

Try update?

Thanks for your replay, findmyway!

I checked my Flux version, and it was indeed v0.9.0.

Then I tried to remove Flux and install it again, and the problem goes away. That’s weird, maybe something got polluted in my old installation which I am not able to find out.