# Flux gradients intepretation

**URL:** <https://discourse.julialang.org/t/flux-gradients-intepretation/58925>\
**Category:** General Usage\
**Tags:** differentiation\
**Created:** [April 9, 2021, 3:19pm UTC](https://discourse.julialang.org/t/flux-gradients-intepretation/58925 "2021-04-09T15:19:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [April 9, 2021, 3:19pm UTC](https://discourse.julialang.org/t/flux-gradients-intepretation/58925/1 "2021-04-09T15:19:03Z")

</div>

There is a code for simple RNN network

```julia
using Flux

rnn = Flux.RNN(2, 3)
seq = [rand(Float32,2) for i = 1:3]
y = [ones(Float32, 3) for i=1:3]

function loss(x, y)
    a = flat_this((rnn.(x) .- y))
   _loss = sum(abs2, a)
end

function flat_this(x)
    arr = ones(0)
    for i=1:length(x)
        arr = vcat(arr, x[i])
    end
    return arr
end

Flux.reset!(rnn);
grads_seq = Flux.gradient(Flux.params(rnn)) do
    sum(loss(seq,y))
end

for p in grads_seq.grads
    println("$p\n")
end

```

Output is the following:

```julia
Pair{Any,Any}(Float32[0.0; 0.0; 0.0], [3.7878030195245587; 1.612768501977888; -2.335516551451906])

Pair{Any,Any}(:(Main.rnn), Base.RefValue{Any}((cell = nothing, state = nothing)))

Pair{Any,Any}(:(Main.y), [[1.8429752588272095, 0.6448161602020264, 2.7944788932800293], [1.6635382175445557, 1.508173942565918, 3.3710970878601074], [3.514638900756836, 1.2005153894424438, 3.7064309120178223]])

Pair{Any,Any}(Float32[0.9722849 -0.80289185; 0.85602915 0.34971967; -0.20451058 -0.39597183], [0.9432474362031119 -0.07124584719900495; -0.6602998565304186 -1.1630237278179147; -4.478283682945728 -4.956677223884434])

Pair{Any,Any}(Float32[0.0, 0.0, 0.0], [0.4786334111414894, -1.9622023618757174, -8.015419969832937])

Pair{Any,Any}(:(Main.seq), [[2.9819661075480615, 0.3152601911834616], [-0.44246049922761166, 0.5381940409583041], [-2.1146017665440304, 1.249914684580733]])

Pair{Any,Any}(Recur(RNNCell(2, 3, tanh)), Base.RefValue{Any}((cell = nothing, state = nothing)))

Pair{Any,Any}(Float32[-0.07491565 0.4195478 0.795933; -0.6760104 -0.46338344 -0.15896535; -0.9976423 -0.2424345 0.89184976], [-0.2417448859956066 -0.27872805151029967 0.9748700622226689; -0.2748194556314892 -1.1553449387338601 1.2234039848773337; -0.3902901267103135 -2.15243746698388 1.8077103083289559])

```

How to understand which weight each gradient are bound to?

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [April 9, 2021, 4:20pm UTC](https://discourse.julialang.org/t/flux-gradients-intepretation/58925/2 "2021-04-09T16:20:25Z")

</div>

You can index the `Grads` struct returned from `gradient` with the parameter arrays in `params(rnn)` (i.e. your model’s parameters) to get the gradient for a specific parameter. e.g. `grads_seq[rnn.cell.Wi]`.