# Lux initialization

**URL:** https://discourse.julialang.org/t/lux-initialization/96049
**Category:** Machine Learning
**Created:** [March 14, 2023, 11:19am UTC](https://discourse.julialang.org/t/lux-initialization/96049 "2023-03-14T11:19:44Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Stefano\_Giampiccolo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefano_giampiccolo/32/46018_2.png) [@Stefano\_Giampiccolo](https://discourse.julialang.org/u/Stefano_Giampiccolo)
#### Post date: [March 14, 2023, 11:19am UTC](https://discourse.julialang.org/t/lux-initialization/96049/1 "2023-03-14T11:19:44Z")

</div>

Hello everyone, I apologize for the probably naive question, but if I wanted to initialize the layer parameters with glorot\_uniform in Lux with a gain smaller than the default, how do I then obtain the parameters and status to define the optimization problem?

I’d like to do something like this:

approximating\_neural\_network = Lux.Chain(Lux.Dense(4, 16, tanh; init\_weight=Lux.glorot\_uniform(rng, 4, 16, gain=0.01), init\_bias=Lux.zeros32), Lux.Dense(16, 16, tanh; init\_weight=Lux.glorot\_uniform(rng, 16, 16, gain=0.01), init\_bias=Lux.zeros32), Lux.Dense(16, 1; init\_weight=Lux.glorot\_uniform(rng, 16, 1, gain=0.01), init\_bias=Lux.zeros32))

# p\_net, st = Lux.setup(rng, approximating\_neural\_network)

p\_net, st?

---

<div class="post-metadata">

### Author: ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)
#### Post date: [March 14, 2023, 12:11pm UTC](https://discourse.julialang.org/t/lux-initialization/96049/2 "2023-03-14T12:11:18Z")

</div>

You want to pass a function as the `init_weight` parameter that is then going to be called like `weight = init_weight(rng, out_dims, in_dims)` from the setup.

This can easily be done by defining your own function calling `glorot_uniform` with the desired gain

```julia
my_glorot_uniform(rng, dims...) = Lux.glorot_uniform(rng, dims...; gain=0.01)

approximating_neural_network = Lux.Chain(
    Lux.Dense(4, 16, tanh; init_weight=my_glorot_uniform, init_bias=Lux.zeros32),
    Lux.Dense(16, 16, tanh; init_weight=my_glorot_uniform, init_bias=Lux.zeros32),
    Lux.Dense(16, 1, tanh; init_weight=my_glorot_uniform, init_bias=Lux.zeros32),
)

p_net, st = Lux.setup(rng, approximating_neural_network)

```

---

<div class="post-metadata">

### Author: ![Stefano\_Giampiccolo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefano_giampiccolo/32/46018_2.png) [@Stefano\_Giampiccolo](https://discourse.julialang.org/u/Stefano_Giampiccolo)
#### Post date: [March 14, 2023, 12:22pm UTC](https://discourse.julialang.org/t/lux-initialization/96049/3 "2023-03-14T12:22:59Z")

</div>

> [@albheim](#):
>
> `my_glorot_uniform`

Thank you!