Issues building Julia 0.7 with Intel MKL on macOS

Internals & Design
1

I see quite a few people were successful building Julia 0.7 using the Intel MKL on Linux. I have tried the same on macOS many times with many variations and pursued many threads, but never managed to do it. I suspect there is a mistake in some make file, or in the julialang instructions page somewhere, though it can also be that I’m doing something really stupid or that my installation is somehow screwed. Since I’ve run out of things to try I resort to the community. Let me explain the problems I have.

I’m using the latest High Sierra 10.13.4, and installed the latest version (2018.2.164) of the free Intel MKL library. I ran xcode-select --install, made a make.user on a fresh clone of julia master containing

USE_INTEL_MKL = 1
USE_INTEL_LIBM = 1

I installed gfortran and cmake with brew (brew install cmake and brew install gcc). I added source /opt/intel/bin/compilervars.sh intel64 -platform mac to my .bash_profile. The script is included with the MKL libraries.

After doing that the environment variable DYLD_LIBRARY_PATH is populated with the following:

/opt/intel/compilers_and_libraries_2018.2.164/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.2.164/mac/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2018.2.164/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.2.164/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.2.164/mac/mkl/lib

The julialang build instructions for macOS indicates that it should be empty, but the build error I get happens also if I reset it to and empty string.

Now, when I do make I first run into the following error

make[1]: *** [/Users/pablo/build/juliamkl/usr/lib/julia/libimf.dylib] Error 1

So, it seems MKL does not include the Intel LIBM. Hence I comment out USE_INTEL_LIBM = 1 in make.user. Then make runs into

make[1]: *** [/Users/pablo/build/juliamkl/usr/lib/julia/libmkl_rt.dylib] Error 1

This file does exist in /opt/intel/mkl/lib, so I suspect something is broken with my path somehow. Since resetting DYLD_LIBRARY_PATH does not help (I get the same error), I try copying by hand /opt/intel/mkl/lib into /Users/pablo/build/juliamkl/usr/lib/julia/. Desperate, I know. That gets me a bit further, but eventually the build fails right at the end with

    LINK usr/lib/julia/sys.dylib
Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.dylib.
make: *** [julia-base-cache] Error 2

That file is actually in /Users/pablo/build/juliamkl/usr/lib/julia/, but linking to it must be failing for some reason.

That is as far as I got. Any pointers would be immensely appreciated!

2 Likes
2

I’m getting similar errors which I only get in v0.7 (for v0.6 everything works fine).

make[1]: *** [/Users/ethananderes/Software/juliaMaster/usr/lib/julia/libmkl_rt.dylib] Error 1
make: *** [julia-base] Error 2
1 Like
3

I had issues with libm when I built MKL on Linux, but they were solved by installing the compiler runtime libs (Intel Developer Zone).

I am not sure if there is something similar for macOS.

4

Were they the same issues above? I did try installing the full Parallel Studio suite, using our institutional license, but it didn’t solve the problems. I suspect a problem somewhere in the make system instead…

5

My issue was that libm was missing from MKL and I could build with USE_INTEL_LIBM = 0. (and after installing the runtime libs I could use USE_INTEL_LIBM = 1)
If you installed the full suite and it did not work, then the issue may be somewhere else as you said. I’m sorry that I couldn’t be more helpful.

1 Like
6

Ok, thanks for the info! So indeed, it seems a macOS-specific thing. And thanks also to EthanAnderes for reproducing this, I was fearing I had screwed up something in my setup.

7

Did you ever end up getting your Julia install working with MKL? I’m currently running into the exact same problem.

8

Hi @Mason, I’m trying precisely now. No luck yet. We’re discussing it also over at https://github.com/JuliaLang/julia/issues/15133#issuecomment-392138864

9

An update: I believe I finally managed to build current master with MKL, see the github issue above.

Now, to my great surprise, I now find much less of a performance advantage of MKL (if any) respect to the new OpenBLAS 3.0 in current master, at least computing a few eigenvalues of a sparse matrix. I have to check more thoroughly, but I’m seeing this kind of pattern for the following quick benchmark

julia> gc(); srand(123); s = sprand(10_000,10_000, 1E-4); @benchmark eigs(s, nev=2)

Julia 6.1 (OpenBLAS 2)

BenchmarkTools.Trial: 
  memory estimate:  4.12 MiB
  allocs estimate:  2089
  --------------
  minimum time:     42.201 ms (0.00% GC)
  median time:      82.471 ms (0.00% GC)
  mean time:        93.641 ms (0.28% GC)
  maximum time:     244.257 ms (0.00% GC)
  --------------
  samples:          54
  evals/sample:     1

JuliaPro 6.2 (with MKL)

BenchmarkTools.Trial:
  memory estimate:  4.09 MiB
  allocs estimate:  1081
  --------------
  minimum time:     18.286 ms (0.00% GC)
  median time:      74.488 ms (0.00% GC)
  mean time:        86.996 ms (0.31% GC)
  maximum time:     235.768 ms (0.00% GC)
  --------------
  samples:          58
  evals/sample:     1

current master (MKL 2018.2.164)

BenchmarkTools.Trial: 
  memory estimate:  4.09 MiB
  allocs estimate:  1391
  --------------
  minimum time:     18.144 ms (0.00% GC)
  median time:      25.967 ms (0.00% GC)
  mean time:        31.886 ms (1.37% GC)
  maximum time:     103.709 ms (0.00% GC)
  --------------
  samples:          157
  evals/sample:     1

current master (OpenBLAS 3.0)

BenchmarkTools.Trial: 
  memory estimate:  4.09 MiB
  allocs estimate:  1400
  --------------
  minimum time:     21.629 ms (0.00% GC)
  median time:      26.397 ms (0.00% GC)
  mean time:        32.254 ms (1.41% GC)
  maximum time:     113.506 ms (1.10% GC)
  --------------
  samples:          155
  evals/sample:     1

I am not 100% sure that I’m actually measuring MKL vs OpenBLAS performance with this test, though. In any case, I’m quite pleased by the performance boost!

10

eigs may not be a good test; most of its time should be spent in native Julia sparse matrix-vector products. Try peakflops or some dense QR factorizations (MKL is reputed to be especially good at the latter, IIRC).

11

Good to know! My main application hinges on eigs of SparseMatrixCSC, so that’s what matters most for me, but here is a test with peakflops, second run

Julia 6.1 (OpenBLAS 2)

julia> peakflops(10000)
4.2177837966089615e10

JuliaPro 6.2 (with MKL)

julia> peakflops(10000)
1.6368673276867664e11

current master (MKL 2018.2.164)

julia> peakflops(10000)
2.0221255393426257e11

current master (OpenBLAS 3.0)

julia> peakflops(10000)
1.8384187931521338e11

Wow!

12

Could you try Matlab & Julia Matrix Operations Benchmark with the different packages?
It is seems there is a big regression in the latest Intel MKL.

13

I’m afraid your benchmarks don’t run on 0.7 for me, they probably need some updating…

14

I was under the impression they are so simple they would work.

Once 0.7 is out I will update them.