Shape of data for sequence learning in Flux.jl?

Specific Domains Machine Learning
1

I’m teaching myself Julia and porting some Python/Keras code over to FluxML, and I’m having trouble shaping the data in Julia.

The python keras code is (basically):

model = Sequential()
model.add(LSTM(9, input_shape=(128,1), return_sequences=True))
model.add(TimeDistributed(Dense(8)))
model.compile(loss='mse', optimizer='nadam')

model.fit(X, Y, epochs=256, .....)

Where X is a sequence of 128 single values (input to the network one at a time), and the output is the resulting 8 outputs at each of the 128 values. So in numpy terms (for 1024 training samples)

X.shape
(1024, 128, 1)
Y.shape
(1024, 128, 8)

Trying to replicate this with Julia, I have the data set up in the same way (using 64 training examples and a seqlen of 32 instead of 128) using nested arrays (is this my issue?), but the following code:

m = Chain(
          Dense(1, 9, NNlib.sigmoid),
          LSTM(9, 9),
          Dense(9, 8))

mynewloss(xs, ys) = Flux.mse.(m(xs), ys)
opt = Flux.ADAM(params(m))
evalcb = () -> @show loss(x_validate, y_validate)

Flux.train!(mynewloss, zip(x_train, y_train), opt, cb = Flux.throttle(evalcb, 30))

produces this error:

DimensionMismatch("second dimension of A, 1, does not match length of x, 32")

more info:

summary(x_train)
"64-element Array{Array{Float32,1},1}"
summary(x_train[1])
"32-element Array{Float32,1}"
summary(x_train[1][1])
"Float32"

summary(y_train)
"64-element Array{Array{Float32,2},1}"
summary(y_train[1])
"32×8 Array{Float32,2}"

Any advice?

2

this means each train sample has 1 number as input, so you’re training against scalar input