Optimization Based on Intel MKL Matrix Multiplication Batch Mode

Internals & Design
1

Intel MKL has a batch mode for Matrix Multiplication (See Introducing Batch GEMM Operations).

It seems it has great potential on speeding up small matrix multiplication operations in 2 ways (I can see, probably more):

  1. Broadcasting
    When multiplying 2 2D array with 3D array and broadcasting the Matrix Multiplication operation along the 3rd dimension.
  2. Lazy Evaluation
    When many small matrix operation are called in a loop and then they are accumulated and sent as a batch job for the batch mode in MKL.

Is this implemented?
If not, could it be added to 1.x (Of course not 1.0, but on its optimization phase once it is released)?

2 Likes
2

This seems to be basically a thin wrapper around threading. It should be supported by good support for easy threading in Julia rather than at the level of MKL.

3

@antoine-levitt,
Indeed if the whole magic is via Multi Threading it should be don in Julia level.
Though it might be less overhead to call one C function instead of multi calls, no?

Unless small matrices will be treated by JuliaBLAS and then engine which will Multi Threaded those operations as above will be the best choice.

4

I don’t think the overhead of calling C is significant. And if you’re doing very small matrices, you’re probably better off with StaticArrays anyway, which bypasses BLAS.

5

TensorOperations.jl maps some tensor operations to BLAS calls already, so you can take advantage of MKL.
https://github.com/Jutho/TensorOperations.jl/blob/6bb64d5c83a890b2ac4e4eddb35af36d0bedf89b/src/implementation/stridedarray.jl

1 Like