Hey. I am trying to model some multimodal time-series data. The data has 3 “components” - a time-series component (shape = [timesteps, channels, batchsize]), and 2 separate flattened components (shape = [numfeatures, batchsize]). This is going to be a regression task.
For my particular data the shapes are [50, 3, 64], [4, 64] and [1, 64]. The output is of shape [12, 64].
I’m a little new to Flux (and Julia in general) and was wondering how one would model this?
My dataloader returns a tuple of xs and ys where xs is also a tuple of (timeseries, flattenedmatrix, flattenedmatrix).
for ((tfw, tfs, tfp), tftgt) in traindl
@show tfw |> size tfs |> size tfp |> size tftgt |> size
break
end
tfw |> size = (50, 3, 64)
tfs |> size = (4, 64)
tfp |> size = (1, 64)
tftgt |> size = (12, 64)
I am a little new to time-series modeling as well so if there’s any resource to help me model time-series well, that would be cool as well. And also any way to add augmentations to time-series is also welcomed as currently I am not adding any data augmentation.
I also cannot for the life of me figure out how to use the LSTM in Flux. Any idea if Self-Attention would be nice thing to try as well?
Thank you!