Simple NLP in Flux with Embedding Layer

New to Julia
2

params in Flux works only with structures that have been made @treelike; this is why it is not catching the embedding layer params. To make it work you need to change it to something like this:

struct EmbeddingLayer
   W
   EmbeddingLayer(mf, vs) = new(param(Flux.glorot_normal(mf, vs)))
end
@Flux.treelike EmbeddingLayer
(m::EmbeddingLayer)(x) = m.W * Flux.onehotbatch(reshape(x, pad_size*N), 0:vocab_size-1)

m = Chain(EmbeddingLayer(max_features, vocab_size),
          x -> reshape(x, max_features, pad_size, N),
          x -> mean(x, dims=2),
          x -> reshape(x, 8, 10),
          Dense(8, 1),
)

The rest of your snippet should stay the same.

Hope this helps.