Linking to MKL 2019 with AMD CPUs?

General Usage Performance
1

Question

Does BinaryBuilder allow Ryzen/Threadripper owners to choose MKL 2019 as the binary to use for MKL.jl and MKLSparse.jl?

Context

Back in 2019 the MKL libraries had a “loophole” that allowed AMD CPUs to take advantage of all the nifty engineering done by Intel by setting the env variable MKL_DEBUG_CPU_TYPE = 5. (In other words, the high MKL performance, it turned out, was not due to Intel hardware, but to their talented software team). That loophole was unfortunately closed by Intel with the 2020 version. It was argued that the reason was that they were adding actual support for AMD. Not so fast…

The great IT guys at my institute recently did a benchmark comparison of different MKL versions on different AMD (Threadripper and Ryzen) and Intel CPUs (i7, i9, Xeon), and found that (1) AMD often smokes Intel CPUs if they can make use of the loophole, and (2) after version 2020 they lost this advantage. Here are the results (an asterisk corresponds to not using MKL_DEBUG_CPU_TYPE = 5).

cbbfgagoaohagipe
cbbfgagoaohagipe766×338 16.8 KB

cgdjmhjchlomnlae
cgdjmhjchlomnlae759×338 16.5 KB

bjdkgkkjomeoldha
bjdkgkkjomeoldha731×340 17.4 KB

cjkojnimojabadap
cjkojnimojabadap772×338 16.9 KB

Hence my question above.

CC: @Bruno_Amorim, @jamblejoe

3 Likes
2

Release MKL-v2019.0.117+2 · JuliaBinaryWrappers/MKL_jll.jl · GitHub, but it may be incompatible with other packages, it’s old

1 Like
3

I cannot answer your question, but maybe it will be possible to use AMDs optimized libraries in the near future with libblastrampoline and Julia v1.7? Did your IT guys test against BLIS/libflame (Basic Linear Algebra Subprograms (BLAS) Libraries - AMD) as well?
Maybe @staticfloat @viralbshah ?

1 Like
4

I just asked, will report back with what they tell me

1 Like
5

Uhm, nice, but how can we use that in combination with the libblastranpoline and MKL.jl? (sorry if this is a dumb question)

6

Manually installing MKL_jll.jl directly with this version (and pinning it, to be safe) and then installing MKL.jl should work I guess.

I.e.

] add MKL_jll@2019.0.117
] pin MKL_jll
] add MKL

UPDATE:

Hm, it fails for me on macOS in a quick test but maybe you have more luck :slight_smile:

1 Like
7

Yes, it fails for me too. It seems to try to inject itself into the sysimg, like in old times (à la 1.6)

8

https://github.com/JuliaLinearAlgebra/MKL.jl/blob/238a4bb8d65037657d1e697f14655834249f309d/Project.toml#L13

1 Like
9

No luck, I tried with MKL_jll = "2019.0.117" in Project.toml and got the following:

(@v1.7) pkg> build MKL
    Building MKL → `~/.julia/dev/MKL/deps/build.log`
┌ Warning: Could not use exact versions of packages in manifest, re-resolving
└ @ Pkg.Operations /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1488
ERROR: Error building `MKL`: 
ERROR: LoadError: InitError: UndefVarError: is_available not defined
Stacktrace:
 [1] getproperty
   @ ./Base.jl:35 [inlined]
 [2] __init__()
   @ MKL ~/.julia/dev/MKL/src/MKL.jl:41
 [3] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:768
 [4] _require_from_serialized(path::String)
   @ Base ./loading.jl:821
 [5] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1130
 [6] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [7] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:451
 [9] top-level scope
   @ none:5
during initialization of module MKL
in expression starting at /Users/pablo/.julia/dev/MKL/deps/build.jl:1
10

Which is precisely the reason why the compat is there in the first place:

Don’t say I didn’t warn you

1 Like
11

Thanks @giordano. Although that’s a no, I’ll mark it as the solution, as I guess the situation is unlikely to change. It’s a pity, but I do understand. The real way forward would be to have Intel open its hand or to have AMD up their game on the software side with BLIS/FLAME. It would be great to have BLIS.jl working with libblastrampoline, though. I think @viralbshah looked into that at some point.

1 Like
12

I mean, if you want to use MKL_jll.jl 2019 and you’re ok with messing with a dev’ed MKL.jl (which you’re already doing) you have also to remove this line:
https://github.com/JuliaLinearAlgebra/MKL.jl/blob/238a4bb8d65037657d1e697f14655834249f309d/src/MKL.jl#L41
for your system (x86_64) MKL_jll.is_available() would be true anyway. I thought that was clear, but probably not.

13

I’m afraid that doesn’t seem to be enough. ] build MKL gets stuck at

(@v1.7) pkg> build MKL
    Building MKL → `~/.julia/dev/MKL/deps/build.log`
┌ Warning: Could not use exact versions of packages in manifest, re-resolving
└ @ Pkg.Operations /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1488
14

I don’t see any errors? Also, in Julia v1.7 building MKL.jl doesn’t do much:
https://github.com/JuliaLinearAlgebra/MKL.jl/blob/238a4bb8d65037657d1e697f14655834249f309d/deps/build.jl#L3-L5
so I’m not sure what you’re expecting exactly.

15

Well, a deved MKL.jl does neither precompile nor build after the recommended changes. It’s not that it gives me a simple warning and finish when I try to build. It just hangs. If I try using MKL without building, it tries to precompile but never finishes either, and actually the REPL crashes if I try to ^C out of precompilation.

Since there is no error, I don’t know what to try next… Thanks in advance for any further ideas!

16

I don’t know, that package is so old that it isn’t even worth the effort trying to make it work if it doesn’t.

As a completely different approach, you can undev MKL.jl, install MKL_jll.jl 2021 regularly and override its artifact, to point to the directory where the artifact for MKL_jll 2019 is: JLL packages · BinaryBuilder.jl

1 Like
17

Have a look at COSMA : Using MKL - Durham University

3 Likes
18

Thanks!