# How to implement the LayerNorm in NeuralPDE by Lux.jl

**URL:** <https://discourse.julialang.org/t/how-to-implement-the-layernorm-in-neuralpde-by-lux-jl/102120>\
**Category:** General Usage\
**Tags:** question, neural-network\
**Created:** [July 26, 2023, 2:06pm UTC](https://discourse.julialang.org/t/how-to-implement-the-layernorm-in-neuralpde-by-lux-jl/102120 "2023-07-26T14:06:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ethan\_Tran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ethan_tran/32/49198_2.png) [@Ethan\_Tran](https://discourse.julialang.org/u/Ethan_Tran)\
**Post date:** [July 26, 2023, 2:06pm UTC](https://discourse.julialang.org/t/how-to-implement-the-layernorm-in-neuralpde-by-lux-jl/102120/1 "2023-07-26T14:06:51Z")

</div>

I tried to implement LayerNorm from Lux.jl package into NeuralPDE following a code:

```julia
chain =[Lux.Chain(Lux.LayerNorm(1,Lux.relu),Dense(1,10,Lux.tanh),Dense(10,20,Lux.tanh),Dense(20,10,Lux.tanh),Dense(10,1)) for _ in 1:12]

```

Then got a message:

```julia
MethodError: no method matching LayerNorm(::Int64, ::typeof(NNlib.relu))

```

The syntax of LayerNorm is:

```julia
LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
          affine::Bool=true, init_bias=zeros32, init_scale=ones32,)

```

But when I implemented the following:

```julia
chain =[Lux.Chain(Lux.LayerNorm((1,1),Lux.relu),Dense(1,10,Lux.tanh),Dense(10,20,Lux.tanh),Dense(20,10,Lux.tanh),Dense(10,1)) for _ in 1:12]

```

The message is:

```julia
MethodError: no method matching layernorm(::Matrix{Float64}, ::Base.ReshapedArray{Float64, 3, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}, ::Base.ReshapedArray{Float64, 3, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}; dims::Colon, epsilon::Float32)

```

Please explant to me the right way to implement it.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [July 28, 2023, 10:49pm UTC](https://discourse.julialang.org/t/how-to-implement-the-layernorm-in-neuralpde-by-lux-jl/102120/2 "2023-07-28T22:49:59Z")

</div>

It should be

```julia
chain =[Lux.Chain(Lux.LayerNorm((1,),Lux.relu),Dense(1,10,Lux.tanh),Dense(10,20,Lux.tanh),Dense(20,10,Lux.tanh),Dense(10,1)) for _ in 1:12]

```

---

<div class="post-metadata">

**Author:** ![Ethan\_Tran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ethan_tran/32/49198_2.png) [@Ethan\_Tran](https://discourse.julialang.org/u/Ethan_Tran)\
**Post date:** [July 28, 2023, 11:42pm UTC](https://discourse.julialang.org/t/how-to-implement-the-layernorm-in-neuralpde-by-lux-jl/102120/3 "2023-07-28T23:42:13Z")

</div>

Thank you for your reply.