Is there a recommended way to one-hot encode a batch of sequences?
More precisely, I am modelling sequences from an alphabet with q letters. A sequence of length N can be one-hot encoded as a q x N one-hot matrix (e.g., using Flux.OneHotMatrix). Then it seems that to encode a batch of B sequences I would need a q x N x B “one-hot tensor”.
What’s the recommended approach?
One possibility is to put the data into a q x (N * B) one-hot matrix, and then reshape this into q x N x B for downstream processing. But this is slow.
I currently put each one hot matrix as a batch and then that matrix into a vector so that the batch is each 1 hot matrix where the N are the sequence steps, and in the loop I restack the data via
x_batch = [Flux.stack([Float32.(x[ii][:,tt]) for ii in 1:length(x)],dims=2) for tt in 1:length(x[1][1,:]) ]
I agree that it is suboptimal and am about to ask a question with more detail provided to get a best practice response hopefully.