Hi all, I would like to initialize the parameters of a Lux.jl neural network with something other than a random distribution; for instance with Lux.glorot_normal. I am using this network with NeuralPDE.jl with the following architecture: dim = length(domains) activation = Lux.σ nnodes = 10 chain …

Initialize Lux.jl NN parameters according to Lux.glorot_normal

avikpal November 15, 2022, 8:10pm 2

You need to specify the initialization for each layer. See Layers - Lux.jl. Including it via setup is not possible since what is being initialized is highly layer dependent.

Topic		Replies	Views
Lux initialization Machine Learning	2	627	March 14, 2023
Impose initialization adn normalization on layers in Flux Machine Learning first-steps	2	742	September 11, 2020
Flux.jl: Initializing Parameters in Specified Range Machine Learning	1	938	July 10, 2020
How to initialize a subset of params of a NN New to Julia flux	2	820	January 16, 2020
Neural Nets training with multiple Chains Lux.jl and CUDA.jl GPU	5	880	October 5, 2022

Initialize Lux.jl NN parameters according to Lux.glorot_normal

Related topics