# CharNN in Flux zoo

**URL:** <https://discourse.julialang.org/t/charnn-in-flux-zoo/37375>\
**Category:** New to Julia\
**Created:** [April 10, 2020, 11:54pm UTC](https://discourse.julialang.org/t/charnn-in-flux-zoo/37375 "2020-04-10T23:54:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![erlebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erlebach/32/12973_2.png) [@erlebach](https://discourse.julialang.org/u/erlebach)\
**Post date:** [April 10, 2020, 11:54pm UTC](https://discourse.julialang.org/t/charnn-in-flux-zoo/37375/1 "2020-04-10T23:54:59Z")

</div>

Hi,

I am running charNN from the Flux Zoo and come across the error in the loss function:

Flux.truncate!() not found. I found out that this function existed in Julia 0.4 and reset the gradients in a model. This function no longer exists and I find no record of it and what one should do. So my question whether Flux takes care of resetting the gradient automatically every time the loss is computed? There is no mention of this in the documentation that I have seen. I am using Julia 1.3 and Flux 0.10.3 . Thank you.

For completeness, I include the entire charNN program.

```julia
using Flux
using Flux: onehot, chunk, batchseq, throttle, crossentropy
using StatsBase: wsample
using Base.Iterators: partition

cd(@ __DIR__ )

isfile("input.txt") ||
  download("https://cs.stanford.edu/people/karpathy/char-rnn/shakespeare_input.txt",
           "input.txt")

text = collect(String(read("input.txt")))
alphabet = [unique(text)..., '_']
# text: 1D array: each element is vector of size=size(alphabet)[1]
text = map(ch -> onehot(ch, alphabet), text)
stop = onehot('_', alphabet)

N = length(alphabet)
seqlen = 50
# Normally, one specifies the batch size
nbatch = 50 # number of batches

# Xs[1:1830][1:50][68,50] or Xs[1:3659][seqlength][length(alphabet),nbatch]
Xs = collect(partition(batchseq(chunk(text, nbatch), stop), seqlen))
Ys = collect(partition(batchseq(chunk(text[2:end], nbatch), stop), seqlen))

m = Chain(
  LSTM(N, 128),
  LSTM(128, 128),
  Dense(128, N),
  softmax)

m = gpu(m)

function loss(xs, ys)
  l = sum(crossentropy.(m.(gpu.(xs)), gpu.(ys)))
  Flux.truncate!(m) 
  return l
end

opt = ADAM(0.01)
tx, ty = (Xs[5], Ys[5])
evalcb = () -> @show loss(tx, ty)

Flux.train!(loss, params(m), zip(Xs, Ys), opt,
            cb = throttle(evalcb, 30))

# Sampling

function sample(m, alphabet, len)
  m = cpu(m)
  Flux.reset!(m)
  buf = IOBuffer()
  c = rand(alphabet)
  for i = 1:len
    write(buf, c)
    c = wsample(alphabet, m(onehot(c, alphabet)).data)
  end
  return String(take!(buf))
end

sample(m, alphabet, 1000) |> println

```

Gordon

---

<div class="post-metadata">

**Author:** ![Owolf](https://avatars.discourse-cdn.com/v4/letter/o/bbce88/32.png) [@Owolf](https://discourse.julialang.org/u/Owolf)\
**Post date:** [April 11, 2020, 7:42pm UTC](https://discourse.julialang.org/t/charnn-in-flux-zoo/37375/2 "2020-04-11T19:42:52Z")

</div>

Hi,  
try `Flux.reset!` , this should work the same way.

Ondrej.