# Help with LSTM on Flux.jl

**URL:** <https://discourse.julialang.org/t/help-with-lstm-on-flux-jl/10976>\
**Category:** General Usage\
**Created:** [May 18, 2018, 5:36am UTC](https://discourse.julialang.org/t/help-with-lstm-on-flux-jl/10976 "2018-05-18T05:36:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rcnlee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rcnlee/32/4107_2.png) [@rcnlee](https://discourse.julialang.org/u/rcnlee)\
**Post date:** [May 18, 2018, 5:36am UTC](https://discourse.julialang.org/t/help-with-lstm-on-flux-jl/10976/1 "2018-05-18T05:36:12Z")

</div>

Hello, I’m trying to figure out what’s wrong with my Flux LSTM model, which is a chain of LSTM, Dense, and softmax. The problem that I’m seeing is that the Dense grads are fine, but I get NaNs for all the LSTM gradients, so the gradients are not propagating back. I don’t get this problem if I replace LSTM with RNN. I’m new to both LSTMs and Flux.jl. I find it strange because LSTM is directly connected to Dense, which is fine.

Here’s the code to reproduce the issue:

```julia
using Flux
using BSON: @load
@load "lstmdata.bson" X Y
m = Chain(LSTM(15, 30), Dense(30, 2), softmax)
function loss(X, Y)
    l = Flux.crossentropy(m.(X)[end],Y[end])
    Flux.truncate!(m)
    return l
end
l=loss(X[1],Y[1])
Flux.back!(l)
W=params(m)
W[6].grad #fine
W[5].grad #NaNs

```

The data BSON can be downloaded here: [lstmdata.bson](https://www.dropbox.com/s/z02bvvdeirhr7op/lstmdata.bson?dl=0). Any help would be greatly appreciated. Thanks!