How to manually update the params in a `Flux.Chain` neural network?

[findmyway]

@xiaodai

function delta(input, i)
  gradient(()->-log(nn(input)[i]), Flux.Params(nn))
end

@xiaodai

opt = ADAM()
Flux.update!(opt, Flux.Params(nn), first_grad )

Optimizers like ADAM use a IdDict internally, so you shouldn’t create new parameters in each update. Just create them once and reuse them when updating.