MKL and libm compile notes: macOS

#1

Just compiled julia with mkl and libm on macOS, ran some tests and it looks to be working fine. It took a while and the info I found online are in various places (mainly from this github issue and the the intel documentation), so here’s the steps in one place for easy reference:

Then, basically the steps listed in the intel documentation:

  1. clone the julia git repo to a local folder:
    git clone https://github.com/JuliaLang/julia.git

  2. check out the latest version, e.g. git checkout v1.1.0

  3. modify Make.inc to use mkl and libm:

change USE_INTEL_MKL from 0 to 1 for mkl
change USE_INTEL_LIBM from 0 to 1 for libm

The default intel mkl install does not have the intel64 subfolder, so change that too:
MKLLIB := $(MKLROOT)/lib/intel64
to
MKLLIB := $(MKLROOT)/lib/
for libm,
LIBM := -L$(MKLROOT)/../compiler/lib/intel64 -limf
to
LIBM := -L$(MKLROOT)/../compiler/lib/ -limf

git diff:

diff --git a/Make.inc b/Make.inc
index e9cb241198..870464aecf 100644
--- a/Make.inc
+++ b/Make.inc
@@ -56,9 +56,9 @@ USE_LLVM_SHLIB := 1
 
 ## Settings for various Intel tools
 # Set to 1 to use MKL
-USE_INTEL_MKL ?= 0
+USE_INTEL_MKL ?= 1
 # Set to 1 to use Intel LIBM
-USE_INTEL_LIBM ?= 0
+USE_INTEL_LIBM ?= 1
 # Set to 1 to enable profiling with Intel VTune Amplifier
 USE_INTEL_JITEVENTS ?= 0
 # Set to 1 to use Intel C, C++, and FORTRAN compilers
@@ -1083,14 +1083,14 @@ endif
 
 ifeq ($(USE_INTEL_LIBM), 1)
 USE_SYSTEM_LIBM := 1
-LIBM := -L$(MKLROOT)/../compiler/lib/intel64 -limf
+LIBM := -L$(MKLROOT)/../compiler/lib/ -limf
 LIBMNAME := libimf
 endif
 
 ifeq ($(USE_INTEL_MKL), 1)
 ifeq ($(USE_BLAS64), 1)
 export MKL_INTERFACE_LAYER := ILP64
-MKLLIB := $(MKLROOT)/lib/intel64
+MKLLIB := $(MKLROOT)/lib/
 else
 MKLLIB := $(MKLROOT)/lib/ia32
 endif

Note that the compilers redistributable libraries by default installs to ~\intel\redist, so further copy/symlink is needed for libm. libm has references to libintlc so need to symlink/copy that too (also, writing to /opt/ so need sudo):

sudo ln -s ~/intel/redist/compilers_and_libraries_2019.2.184/mac/compiler/lib/libimf.dylib /opt/intel/lib/libimf.dylib
sudo ln -s ~/intel/redist/compilers_and_libraries_2019.2.184/mac/compiler/lib/libintlc.dylib /opt/intel/lib/libintlc.dylib

change compilers_and_libraries_2019.2.184 to the installed version if different.

  1. Now run the build, will take a while.

cd to the git repo cloned and

source /opt/intel/mkl/bin/mklvars.sh intel64 ilp64
then make

  1. The source statement above setup the env variable but julia may need further sym links, from the cloned git repo directory, do
ln -s /opt/intel/mkl/lib/libmkl_rt.dylib usr/lib/julia/libmkl_rt.dylib
ln -s /opt/intel/lib/libimf.dylib usr/lib/julia/libimf.dylib 
ln -s /opt/intel/lib/libintlc.dylib usr/lib/julia/libintlc.dylib

Note the lack of / in front of usr: it symlinks to the usr folder in the current directory (cloned julia repo directory), not the /usr/ folder on root

Now, julia should be compiled and you may run it as ./julia from the current directory (or make install; run with make install -n to see which files will be copied where). You may need to recompile some packages, such as Pkg.build("SpecialFunctions").

6 Likes

#2

Note, however, libm lacks certain features and Base.runtests("math") may complain “could not find function __ldexp_exp in library libimf”, see here

0 Likes

#3

You shouldn’t have to edit Make.inc. Did the steps provided in Julia’s README.md not work for MKL?

Note that the make ?= syntax means: set this variable if it doesn’t already have a value. So you should be able to do stuff like USE_INTEL_MKL=1 make (or just put USE_INTEL_MKL=1 in Make.user as described in README.md). And source /path/to/intel/bin/compilervars.sh intel64 should take care of setting MKLROOT.

1 Like

#4

Thanks! yes, USE_INTEL_MKL = 1 can go in Make.user; However,

  • compilervars.sh is now renamed to mklvars.sh, following the current link to intel mkl;
  • it’s not about setting $MKLROOT: on macs, the library files live in $(MKLROOT)/lib/ and not $(MKLROOT)/lib/intel64, (there are no intel64 folders in lib) therefore needs to be changed
> cd $MKLROOT/lib
> pwd
/opt/intel/compilers_and_libraries_2019.2.184/mac/mkl/lib
> ls -halt
total 2375616
-rwxr-xr-x   1 root  wheel    30M Feb 20 11:35 libmkl_tbb_thread.dylib
drwxr-xr-x  31 root  wheel   992B Feb 20 11:35 .
-rw-r--r--   1 root  wheel    32M Feb 20 11:35 libmkl_tbb_thread.a
-rw-r--r--   1 root  wheel   6.3M Feb 20 11:35 libmkl_lapack95_lp64.a
-rw-r--r--   1 root  wheel   6.4M Feb 20 11:35 libmkl_lapack95_ilp64.a
-rw-r--r--   1 root  wheel   542K Feb 20 11:35 libmkl_blas95_lp64.a
-rw-r--r--   1 root  wheel   545K Feb 20 11:35 libmkl_blas95_ilp64.a
drwxr-xr-x   9 root  wheel   288B Feb 20 11:35 ..
drwxr-xr-x   3 root  wheel    96B Feb 20 11:35 locale
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc3.dylib
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc2.dylib
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx512.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx2.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx.dylib
-rwxr-xr-x   1 root  wheel    25M Feb 20 11:35 libmkl_sequential.dylib
-rw-r--r--   1 root  wheel    23M Feb 20 11:35 libmkl_sequential.a
-rwxr-xr-x   1 root  wheel   9.7M Feb 20 11:35 libmkl_rt.dylib
-rwxr-xr-x   1 root  wheel    46M Feb 20 11:35 libmkl_mc3.dylib
-rwxr-xr-x   1 root  wheel    44M Feb 20 11:35 libmkl_mc.dylib
-rwxr-xr-x   1 root  wheel    44M Feb 20 11:35 libmkl_intel_thread.dylib
-rw-r--r--   1 root  wheel    44M Feb 20 11:35 libmkl_intel_thread.a
-rwxr-xr-x   1 root  wheel    21M Feb 20 11:35 libmkl_intel_lp64.dylib
-rw-r--r--   1 root  wheel    32M Feb 20 11:35 libmkl_intel_lp64.a
-rwxr-xr-x   1 root  wheel    17M Feb 20 11:35 libmkl_intel_ilp64.dylib
-rw-r--r--   1 root  wheel    28M Feb 20 11:35 libmkl_intel_ilp64.a
-rwxr-xr-x   1 root  wheel    64M Feb 20 11:35 libmkl_core.dylib
-rw-r--r--   1 root  wheel   442M Feb 20 11:35 libmkl_core.a
-rwxr-xr-x   1 root  wheel    65M Feb 20 11:34 libmkl_avx512.dylib
-rwxr-xr-x   1 root  wheel    58M Feb 20 11:34 libmkl_avx2.dylib
-rwxr-xr-x   1 root  wheel    50M Feb 20 11:34 libmkl_avx.dylib
0 Likes

#5

You can instead specify these in a Make.user file.
Also, a couple warnings:

  1. MKL is incompatible with ARPACK, a dependency of packages such as Distributions and LightGraphs.
  2. SpecialFunctions.jl can take advantage of openlibm, but not intellibm, for much better performance. Not as severe as the above, because SpecialFunctions will still work. It can however be many times slower.
0 Likes

LAPACK multithreading
#6

I have a follow-up work-in-progress work through here

0 Likes