Yet another failed attempt to build julia with MKL

General Usage
1

I’ve reached bottom and despair…

I’ve tried MANY times to build julia with MKL. I’ve done all sorts of gimmicks and tricks people mention here at GitHub and all over the internet. Repetitive failure each time more or less with a different error.

My machine:

MBP 15-inch Early 2017
2.9GHz Inter Core i7
macOS High Sierra 10.13.1

with:
MKL version 2018 Update 1

I document here my latest attempt:
.zshrc:

source /opt/intel/mkl/bin/mklvars.sh intel64 ilp64
export TBBROOT=fdsljkfds

Make.user:

  USEICC = 0
  USEIFC = 0
  USE_INTEL_MKL = 1
  USE_INTEL_MKL_FFT = 1
  USE_INTEL_LIBM = 1
  MKLROOT = /opt/intel/compilers_and_libraries/mac/mkl
  LIBM = /opt/intel/compilers_and_libraries/mac/lib -limf

Then I removed the /intel64 as well as some if statements from Make.inc as described at #18167. My Make.inc reads now:

 988 # Intel libraries
 989
 990 ifeq ($(USE_INTEL_LIBM), 1)
 991 USE_SYSTEM_LIBM := 1
 992 LIBM := -L$(MKLROOT)/../compiler/lib -limf
 993 LIBMNAME := libimf
 994 endif
 995
 996 ifeq ($(USE_INTEL_MKL), 1)
 997 # ifeq ($(USE_BLAS64), 1)
 998 export MKL_INTERFACE_LAYER := ILP64
 999 MKLLIB := $(MKLROOT)/lib
1000 # else
1001 # MKLLIB := $(MKLROOT)/lib/ia32
1002 # endif
1003 USE_SYSTEM_BLAS:=1
1004 USE_SYSTEM_LAPACK:=1
1005 # ifeq ($(OS), WINNT)
1006 # LIBBLASNAME := mkl_rt
1007 # LIBLAPACKNAME := mkl_rt
1008 # # work around libtool issue with arpack
1009 # MKL_LDFLAGS := -L$(MKLLIB) -Wl,-lmkl_rt
1010 # else
1011 LIBBLASNAME := libmkl_rt
1012 LIBLAPACKNAME := libmkl_rt
1013 MKL_LDFLAGS := -L$(MKLLIB) -lmkl_rt
1014 # endif
1015 ifneq ($(strip $(MKLLIB)),)
1016   ifeq ($(OS), Linux)
1017     RPATH_MKL := -Wl,-rpath,$(MKLLIB)
1018     RPATH     += $(RPATH_MKL)
1019     MKL_LDFLAGS += $(RPATH_MKL)
1020   endif
1021 endif
1022 LIBBLAS   := $(MKL_LDFLAGS)
1023 LIBLAPACK := $(MKL_LDFLAGS)
1024 endif
1025
1026 ifeq ($(USE_INTEL_MKL_FFT), 1)
1027 USE_SYSTEM_FFTW := 1
1028 LIBFFTWNAME := libmkl_rt
1029 LIBFFTWFNAME := libmkl_rt
1030 endif

I create folder usr/lib and link files:

 ln -s /opt/intel/lib/libimf.dylib usr/lib/libimf.dylib
 ln -s /opt/intel/mkl/lib/libmkl_rt.dylib usr/lib/libmkl_rt.dylib
 ln -s /opt/intel/lib/libintlc.dylib usr/lib/libintlc.dylib

Then, running make gives (this time):

  CC       libcurl_la-connect.lo
/Users/navid/julia/deps/srccache/curl-7.53.1/lib/connect.c:1087:12: error: 'connectx' is only available on macOS
      10.11 or newer [-Werror,-Wunguarded-availability]
      rc = connectx(sockfd, &endpoints, SAE_ASSOCID_ANY,
           ^~~~~~~~
/usr/include/sys/socket.h:713:5: note: 'connectx' has been explicitly marked partial here
int connectx(int, const sa_endpoints_t *, sae_associd_t, unsigned int,
    ^
/Users/navid/julia/deps/srccache/curl-7.53.1/lib/connect.c:1087:12: note: enclose 'connectx' in a
      __builtin_available check to silence this warning
      rc = connectx(sockfd, &endpoints, SAE_ASSOCID_ANY,
           ^~~~~~~~
1 error generated.
make[4]: *** [libcurl_la-connect.lo] Error 1
make[3]: *** [all] Error 2
make[2]: *** [all-recursive] Error 1
make[1]: *** [scratch/curl-7.53.1/build-compiled] Error 2
make: *** [julia-deps] Error 2
1 Like
2

Which version/commit of Julia are you trying to build?

3

I can replicate with the latest julia master on osx 10.13.2.
( git rev-parse HEAD gives me b385693ceb66458849dee5267fc56e051e517d95)

It would be much appreciated if someone who actually managed to build with MKL on osx could document this step by step. I’m willing to assist, of course.

Meanwhile, I just saw that the free JuliaPro is now available with MKL. Makes me wonder if the julialang one might also be distributed with MKL?

For the curious, it seems that “MKL destroys OpenBLAS in julia”, that’s why people keep trying to build it.

4

Destroys was probably a little strong, but yes there are some cases where it’s much faster with not many the opposite :stuck_out_tongue:.

It’s mentioned in that thread that I believe it can’t due to licensing issues (because there’s still some GPL in the standard libraries?). Someone else can probably make that more precise.

5

I did a few tests with an old AMD processor, and OpenBLAS often outperformed MKL on it. Intel does warn that their software is only optimized for Intel hardware.
I have since upgraded to a computer with a Ryzen processor. Ryzen is apparently a lot like Haswell, so perhaps MKL will perform better on it. But I wouldn’t be surprised if MKL doesn’t recognize Ryzen and thus falls back to generic kernels instead of Haswell.

I could try in a few days.

1 Like
6

Just to throw some older tests in here:

[Album] imgur.com

The biggest difference seems to be if you solve a bunch of linear systems. This is noticeable for solving stiff ODEs BTW.

7

What matrix operation are you using? For most jobs the time advantage of MKL is not that noticeable, especially if you are using non-intel core. Eigen3 and Juia can handle most matrix operations for me.