Flux LSTM format of input train data

General Usage
, ,
1

Hello community,

I am trying to create easy example using LSTM model. The idea is that model should predict sinus signal. I expected that it is really easy task and it will be done in second. However I am still not able to find good way that would do at least something meaningful. Here is what I recently tried:

train_data  = [sin.(0+i:0.1:1+i) for i in 0:num_samples]
train_labels  = [sin.(0+i+1:0.1:1+i+1) for i in 0:num_samples]

m = Chain(LSTM(1,16), LSTM(16,1))

function loss(x, y)
    out = sum(Flux.mse.(m.(x), y))
    Flux.reset!(m)
    out
end

ps = params(m)

function evalcb()
     @show(sum(loss.(train_data, train_labels)))
 end

opt = ADAM();

@epochs 100 Flux.train!(loss, ps, zip(train_data, train_labels), opt, cb = Flux.throttle(evalcb, 60))

This will make a model that is not able to predict sinus function :smiley:

However I would like to firstly ask if I use good input data? Is it type Array{Array{Float64,1},1} correct? Is it correct way how to create a sequence of data?

The second things is when I call a model like:

m.(train_data[1])

Why it returns Array{Array{Float64,2},1}? I mean why LSTM returns 2 dimensional array?

I have already went through lot of information about LSTM but practical use in Julia is something I do not understand at all.

After all, non - reccurent networks works pretty good in Flux :slight_smile:

I will be really happy if somebody can give me some hint what to do.

Thank you.