# The right way to implement a residual network (ResNet)

**URL:** <https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937>\
**Category:** Machine Learning\
**Tags:** question, flux\
**Created:** [August 24, 2021, 7:51pm UTC](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937 "2021-08-24T19:51:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![weiqi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weiqi/32/22148_2.png) [@weiqi](https://discourse.julialang.org/u/weiqi)\
**Post date:** [August 24, 2021, 7:51pm UTC](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937/1 "2021-08-24T19:51:20Z")

</div>

I am not sure if the following implementation of ResNet is correct. Any suggestion?

What I am not sure are:  
(1) for the network of nn\_x, will the ResNet block sm share parameters? I don’t expect they share the same parameters.

```julia
nodes = 32
nq = 32

m = Chain(Dense(nodes, nodes, mish),
          Dense(nodes, nodes, mish),
          Dense(nodes, nodes, mish))

sm = SkipConnection(m, +);

# TODO: use iterator to construct NN
nn_x = Chain(Dense(n_var_in - 1, nodes, mish),
           sm, sm, sm, sm, sm, sm, sm, sm, sm, sm,
           Dense(nodes, nq * n_var_out)) |> gpu

nn_t = Chain(Dense(1, nodes, mish),
            sm, sm, sm, sm, sm, sm, sm, sm, sm, sm,
            Dense(nodes, nq)) |> gpu

p_all = Flux.params(nn_x, nn_t)

function pred(x)
    Gu = reshape(nn_x(x[2:end, :]), n_var_out, nq, :);
    tq = nn_t(x[1:1, :])
    Gutq = hcat([Gu[:, :, i] * tq[:, i] for i in 1:size(x)[2]]...)
    return Gutq
end

```

---

<div class="post-metadata">

**Author:** ![DrChainsaw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drchainsaw/32/8497_2.png) [@DrChainsaw](https://discourse.julialang.org/u/DrChainsaw)\
**Post date:** [August 25, 2021, 3:51pm UTC](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937/2 "2021-08-25T15:51:25Z")

</div>

They will all share params. Calling `Dense(...)` is what initializes the params, so what happens is that you model will just pass the output from `sm` into `sm` again a few times. You might want to put the `m=...` and `sm=...` lines of your example in a function and call it multiple times to get independent instances.

I haven’t scrutinised the rest of the code, but it looks ok to me at a glance.

---

<div class="post-metadata">

**Author:** ![scimas](https://avatars.discourse-cdn.com/v4/letter/s/a3d4f5/32.png) [@scimas](https://discourse.julialang.org/u/scimas)\
**Post date:** [August 25, 2021, 5:14pm UTC](https://discourse.julialang.org/t/the-right-way-to-implement-a-residual-network-resnet/66937/3 "2021-08-25T17:14:24Z")

</div>

[https://github.com/FluxML/Metalhead.jl/blob/master/src/resnet.jl](https://github.com/FluxML/Metalhead.jl/blob/master/src/resnet.jl)

Might want to use this or use as a reference.