MKL and libm compile notes: macOS

Egwene_al_Vere · February 20, 2019, 9:48pm

Just compiled julia with mkl and libm on macOS, ran some tests and it looks to be working fine. It took a while and the info I found online are in various places (mainly from this github issue and the the intel documentation), so here’s the steps in one place for easy reference:

mkl: Accelerate Fast Math with Intel® oneAPI Math Kernel Library
libm: not included in mkl for mac, it’s in Compilers Redistributable Libraries: Intel® C++ and Fortran Compilers Redistributable Libraries by Version
(I used [Parallel Studio XE 2019 (All Editions)], c++ compiler)

Then, basically the steps listed in the intel documentation:

clone the julia git repo to a local folder:
git clone https://github.com/JuliaLang/julia.git
check out the latest version, e.g. git checkout v1.1.0
modify Make.inc to use mkl and libm:

change USE_INTEL_MKL from 0 to 1 for mkl
change USE_INTEL_LIBM from 0 to 1 for libm

The default intel mkl install does not have the intel64 subfolder, so change that too:
MKLLIB := $(MKLROOT)/lib/intel64
to
MKLLIB := $(MKLROOT)/lib/
for libm,
LIBM := -L$(MKLROOT)/../compiler/lib/intel64 -limf
to
LIBM := -L$(MKLROOT)/../compiler/lib/ -limf

git diff:

diff --git a/Make.inc b/Make.inc
index e9cb241198..870464aecf 100644
--- a/Make.inc
+++ b/Make.inc
@@ -56,9 +56,9 @@ USE_LLVM_SHLIB := 1
 
 ## Settings for various Intel tools
 # Set to 1 to use MKL
-USE_INTEL_MKL ?= 0
+USE_INTEL_MKL ?= 1
 # Set to 1 to use Intel LIBM
-USE_INTEL_LIBM ?= 0
+USE_INTEL_LIBM ?= 1
 # Set to 1 to enable profiling with Intel VTune Amplifier
 USE_INTEL_JITEVENTS ?= 0
 # Set to 1 to use Intel C, C++, and FORTRAN compilers
@@ -1083,14 +1083,14 @@ endif
 
 ifeq ($(USE_INTEL_LIBM), 1)
 USE_SYSTEM_LIBM := 1
-LIBM := -L$(MKLROOT)/../compiler/lib/intel64 -limf
+LIBM := -L$(MKLROOT)/../compiler/lib/ -limf
 LIBMNAME := libimf
 endif
 
 ifeq ($(USE_INTEL_MKL), 1)
 ifeq ($(USE_BLAS64), 1)
 export MKL_INTERFACE_LAYER := ILP64
-MKLLIB := $(MKLROOT)/lib/intel64
+MKLLIB := $(MKLROOT)/lib/
 else
 MKLLIB := $(MKLROOT)/lib/ia32
 endif

Note that the compilers redistributable libraries by default installs to ~\intel\redist, so further copy/symlink is needed for libm. libm has references to libintlc so need to symlink/copy that too (also, writing to /opt/ so need sudo):

sudo ln -s ~/intel/redist/compilers_and_libraries_2019.2.184/mac/compiler/lib/libimf.dylib /opt/intel/lib/libimf.dylib
sudo ln -s ~/intel/redist/compilers_and_libraries_2019.2.184/mac/compiler/lib/libintlc.dylib /opt/intel/lib/libintlc.dylib

change compilers_and_libraries_2019.2.184 to the installed version if different.

Now run the build, will take a while.

cd to the git repo cloned and

source /opt/intel/mkl/bin/mklvars.sh intel64 ilp64
then make

The source statement above setup the env variable but julia may need further sym links, from the cloned git repo directory, do

ln -s /opt/intel/mkl/lib/libmkl_rt.dylib usr/lib/julia/libmkl_rt.dylib
ln -s /opt/intel/lib/libimf.dylib usr/lib/julia/libimf.dylib 
ln -s /opt/intel/lib/libintlc.dylib usr/lib/julia/libintlc.dylib

Note the lack of / in front of usr: it symlinks to the usr folder in the current directory (cloned julia repo directory), not the /usr/ folder on root

Now, julia should be compiled and you may run it as ./julia from the current directory (or make install; run with make install -n to see which files will be copied where). You may need to recompile some packages, such as Pkg.build("SpecialFunctions").

Egwene_al_Vere · February 20, 2019, 10:38pm

Note, however, libm lacks certain features and Base.runtests("math") may complain “could not find function __ldexp_exp in library libimf”, see here

tkoolen · February 20, 2019, 10:59pm

You shouldn’t have to edit Make.inc. Did the steps provided in Julia’s README.md not work for MKL?

Note that the make ?= syntax means: set this variable if it doesn’t already have a value. So you should be able to do stuff like USE_INTEL_MKL=1 make (or just put USE_INTEL_MKL=1 in Make.user as described in README.md). And source /path/to/intel/bin/compilervars.sh intel64 should take care of setting MKLROOT.

Egwene_al_Vere · February 20, 2019, 11:10pm

Thanks! yes, USE_INTEL_MKL = 1 can go in Make.user; However,

compilervars.sh is now renamed to mklvars.sh, following the current link to intel mkl;
it’s not about setting $MKLROOT: on macs, the library files live in $(MKLROOT)/lib/ and not $(MKLROOT)/lib/intel64, (there are no intel64 folders in lib) therefore needs to be changed

> cd $MKLROOT/lib
> pwd
/opt/intel/compilers_and_libraries_2019.2.184/mac/mkl/lib
> ls -halt
total 2375616
-rwxr-xr-x   1 root  wheel    30M Feb 20 11:35 libmkl_tbb_thread.dylib
drwxr-xr-x  31 root  wheel   992B Feb 20 11:35 .
-rw-r--r--   1 root  wheel    32M Feb 20 11:35 libmkl_tbb_thread.a
-rw-r--r--   1 root  wheel   6.3M Feb 20 11:35 libmkl_lapack95_lp64.a
-rw-r--r--   1 root  wheel   6.4M Feb 20 11:35 libmkl_lapack95_ilp64.a
-rw-r--r--   1 root  wheel   542K Feb 20 11:35 libmkl_blas95_lp64.a
-rw-r--r--   1 root  wheel   545K Feb 20 11:35 libmkl_blas95_ilp64.a
drwxr-xr-x   9 root  wheel   288B Feb 20 11:35 ..
drwxr-xr-x   3 root  wheel    96B Feb 20 11:35 locale
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc3.dylib
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc2.dylib
-rwxr-xr-x   1 root  wheel    11M Feb 20 11:35 libmkl_vml_mc.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx512.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx2.dylib
-rwxr-xr-x   1 root  wheel    12M Feb 20 11:35 libmkl_vml_avx.dylib
-rwxr-xr-x   1 root  wheel    25M Feb 20 11:35 libmkl_sequential.dylib
-rw-r--r--   1 root  wheel    23M Feb 20 11:35 libmkl_sequential.a
-rwxr-xr-x   1 root  wheel   9.7M Feb 20 11:35 libmkl_rt.dylib
-rwxr-xr-x   1 root  wheel    46M Feb 20 11:35 libmkl_mc3.dylib
-rwxr-xr-x   1 root  wheel    44M Feb 20 11:35 libmkl_mc.dylib
-rwxr-xr-x   1 root  wheel    44M Feb 20 11:35 libmkl_intel_thread.dylib
-rw-r--r--   1 root  wheel    44M Feb 20 11:35 libmkl_intel_thread.a
-rwxr-xr-x   1 root  wheel    21M Feb 20 11:35 libmkl_intel_lp64.dylib
-rw-r--r--   1 root  wheel    32M Feb 20 11:35 libmkl_intel_lp64.a
-rwxr-xr-x   1 root  wheel    17M Feb 20 11:35 libmkl_intel_ilp64.dylib
-rw-r--r--   1 root  wheel    28M Feb 20 11:35 libmkl_intel_ilp64.a
-rwxr-xr-x   1 root  wheel    64M Feb 20 11:35 libmkl_core.dylib
-rw-r--r--   1 root  wheel   442M Feb 20 11:35 libmkl_core.a
-rwxr-xr-x   1 root  wheel    65M Feb 20 11:34 libmkl_avx512.dylib
-rwxr-xr-x   1 root  wheel    58M Feb 20 11:34 libmkl_avx2.dylib
-rwxr-xr-x   1 root  wheel    50M Feb 20 11:34 libmkl_avx.dylib

Elrod · February 21, 2019, 2:04am

You can instead specify these in a Make.user file.
Also, a couple warnings:

MKL is incompatible with ARPACK, a dependency of packages such as Distributions and LightGraphs.
SpecialFunctions.jl can take advantage of openlibm, but not intellibm, for much better performance. Not as severe as the above, because SpecialFunctions will still work. It can however be many times slower.

Egwene_al_Vere · February 22, 2019, 3:01am

I have a follow-up work-in-progress work through here

carstenbauer · April 18, 2020, 12:24am

Is this still the best “guide” for compiling Julia with MKL on macOS? I’m a bit surprised that this is still so complicated in 2020. Why haven’t, for example, these path changes been merged?

(Unfortunately, the instructions above don’t work for me, see Building Julia with Intel MKL and Intel LIBM on macOS )

Topic		Replies	Views
Building Julia with Intel MKL and Intel LIBM on macOS General Usage	6	1803	April 18, 2020
Issues building Julia 0.7 with Intel MKL on macOS Internals & Design build	13	2725	June 13, 2018
Problems Installing Julia and MKL OSX Performance mkl	6	1505	November 19, 2018
Yet another failed attempt to build julia with MKL General Usage	6	3391	December 25, 2017
Building Julia 0.6.0 with MKL on OSX 10.12.5 General Usage mkl	8	2443	July 30, 2017

MKL and libm compile notes: macOS

Related topics