I’m sort of struggling with the same issue as here , but the answer isn’t really clear to me.
What I’m trying to do is to make a basic timeseries forecasting with a RNN architecture with minibatches.
I got the minibatches working for a Dense model, but the RNN seems to work different.
As data, I have sequences of length S, and I have N observations. For a simple dense model (
Chain(Dense(S, 10), Dense(10, 1)) I need to structure my data as a SxN Array. After that, I used
Flux.Data.DataLoader to create batches.
So far, this works. But now the RNN: simply changing the model to
Chain(RNN(S,10), Dense(10,1)) does not work as expected.
Well, I can create a model
m and feed it
m(train_x) with train_x having SxN dimensions. That returns an 1xN output.
However, running this again on a subset with dimensions SxM with M a different dimension as N, returns a DimensionMismatch error.
What I think is going on, is that the RNN needs a specification for the amount of features (which is 1 in my case) and is assuming that I have 210 features. So, this suggests that to use batches, I would need to add an extra dimension for the features. This is similar to what I have seen in tensorflow.
I have also looked at the char-rnn.jl example from the Flux model-zoo, but I just don’t get it to work. Tried different dimensions, reshaping, etc etc. I was hoping someone could either explain what is going on there, or what the dimensions ought be.