Flux.jl: Initializing Parameters in Specified Range

Specific Domains Machine Learning
1

Hello! I hope you’re all doing well.

I’m trying to figure out how to initialize the parameters of a simple Dense layer object uniformly in a range that I specify. I know that I can access how the layer parameters are initialized with initW and initb, but I don’t know what do after that. Any ideas?

Thanks in advance!

2

If all you need is a feed-forward neural network on a mid-scale dataset you can use the BetaML library (disclaimer: I’m the author):

using Distributions
l1   = DenseLayer(23,15,f=sigmoid, w=rand(Uniform(-2,2),15,23), wb=rand(Uniform(-2,2),15))  # Activation function is ReLU
l2   = DenseLayer(15,1,f=identity, w=rand(Uniform(-2,2),1,15), wb=rand(Uniform(-2,2),1))
mynn = buildNetwork([l1,l2],squaredCost,name="Bike sharing regression model") # Build the NN and use the squared cost (aka MSE) as error function
Nn.show(mynn)
"""
*** Bike sharing regression model (2 layers, non trained)

#    # In    # Out   Type
1    23          15          DenseLayer 
2    15          1       DenseLayer 
"""
# Training it (default to ADAM)
# xtrain: (n,d) -  ytrain: (n,1)
res   = train!(mynn,xtrain,ytrain,epochs=100,batchSize=8,optAlg=ADAM(),verbosity=HIGH) # Use optAlg=SGD() to get Stochastic Gradient Descent
ŷtest = predict(mynn,xtrain)