Multiply many-matrices by many-vectors

I guess this convention came from deep learning frameworks. E.g pytorch has some batched operations, like bmm, which do matrix multiplication for A and B, where A and B are vectors of matrix represented as a rank-3 tensor (in Python they treat the first dimension as batch dimension)

Batched operation here means do the same operation on a batch of objects (matrix, vectors, etc.) This is extremely useful when you do parallel computing or programming some GPU code since the batch dimensions can be easily paralleled.

E.g for batched matrix multiplication, cublas has gemmBatched routine for vectors of matrix, and gemmBatchedStrided for rank-3 tensors.

1 Like