Intel Mkl Windows building

I am not sure. I don’t know how to use this repository or fix its issues. If you guys know please submit a pull request to update the repository.
I believe this repository should also provide prebuilt releases for the convenience of users (especially because JuliaPro doesn’t provide MKL anymore).

What issues? It seems ]add MKL and then using MKL works fine on Windows? I didn’t rebuild my sysimage for it but you’d do it just like how you’d do it for any other package.

2 Likes

By issues I mean the delay in REPL.
I am gonna try that.

The delay in the REPL isn’t that much - it just means that it takes a second or two longer to start when first running Julia.

Just to explain what I think MKL.jl does:

add MKL downloads the Intel binaries, modifies one of the files in your Julia install to tell it to use the MKL, and then rebuilds your sysimg. After doing this you are always using the MKL (even without the using statement). Uninstalling the package won’t remove the MKL (except maybe the Intel binaries - bad things might happen since Julia still looks for them).

using MKL just seems to set some threading options. I’m not sure this is entirely necessary (I never used to bother).

The only thing that needs to happen to get a fast REPL start-up is for an extra file to be included during the sysimg rebuild. (The problem is that file isn’t included in the standard Julia install.)

3 Likes

]add MKL
result:

ERROR: The following package names could not be resolved:
 * MKL (not found in project, manifest or registry)
Please specify by known `name=uuid`

Try

]add https://github.com/JuliaComputing/MKL.jl

Hopefully that works - don’t have my laptop to check.

1 Like

This took some time to complete, and seemed to do something.

(v1.1) pkg> add https://github.com/JuliaComputing/MKL.jl
  Updating registry at `C:\Users\username\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `https://github.com/JuliaComputing/MKL.jl`
  Updating git-repo `https://github.com/JuliaComputing/MKL.jl` Resolving package versions...
 Installed HTTPClient ────── v0.2.1
 Installed Libz ──────────── v1.0.0
 Installed WinRPM ────────── v0.4.2
 Installed LibCURL ───────── v0.5.1
 Installed TextWrap ──────── v0.3.0
 Installed BufferedStreams ─ v1.0.0
 Installed PackageCompiler ─ v0.6.3
 Installed WebIO ─────────── v0.8.4
 Installed LibExpat ──────── v0.5.0
 Installed BinDeps ───────── v0.8.10
 Installed ArgParse ──────── v0.6.2
  Updating `C:\Users\username\.julia\environments\v1.1\Project.toml`
  [33e6dc65] + MKL v0.0.0 #master (https://github.com/JuliaComputing/MKL.jl)
  Updating `C:\Users\username\.julia\environments\v1.1\Manifest.toml`
  [c7e460c6] + ArgParse v0.6.2
  [9e28174c] + BinDeps v0.8.10
  [e1450e63] + BufferedStreams v1.0.0
  [0862f596] + HTTPClient v0.2.1
  [b27032c2] + LibCURL v0.5.1
  [522f3ed2] + LibExpat v0.5.0
  [2ec943e9] + Libz v1.0.0
  [33e6dc65] + MKL v0.0.0 #master (https://github.com/JuliaComputing/MKL.jl)
  [9b87118b] + PackageCompiler v0.6.3
  [b718987f] + TextWrap v0.3.0
  [0f1e0344] ↑ WebIO v0.8.1 ⇒ v0.8.4
  [c17dfb99] + WinRPM v0.4.2
  Building WebIO ──────────→ `C:\Users\username\.julia\packages\WebIO\kk2q1\deps\build.log`
  Building LibCURL ────────→ `C:\Users\username\.julia\packages\LibCURL\teHPG\deps\build.log`
  Building WinRPM ─────────→ `C:\Users\username\.julia\packages\WinRPM\Y9QdZ\deps\build.log`
  Building PackageCompiler → `C:\Users\username\.julia\packages\PackageCompiler\oT98U\deps\build.log`
  Building MKL ────────────→ `C:\Users\username\.julia\packages\MKL\36eeJ\deps\build.log`

However, when I ran this code after it still says it is openBlas.

using LinearAlgebra
 BLAS.vendor()

and now when I restart my Julia this is the result:
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll.

Could you explain what do you mean by: The loss of arpack and suitesparse functionality will take more time, but there is progress.?

1 Like

As far as I understand it, if you use mkl.jl and package compile it into your system image, it works great for almost all of base and is a solid experience.

The current issue is that other packages which assume that openblas is installed need to be patched to support using mkl for any dense calculations.

In particular, https://docs.julialang.org/en/v1/stdlib/SparseArrays/index.html uses suitesparse as matrix factorization lfor linear solves (I think) and arpack.jl currently requires openblas (necessary for eigs).People are trying to work through the binary builder issues to make usage with mkl seamless,but I would guess later in July

1 Like

Yes, this what should happen; the new sysimg will only take effect once Julia is restarted.

That means the required dlls are not on the Windows path. You need to add the location that it was installed to your path - for me that is C:\Users\dawbarton\.julia\packages\MKL\36eeJ\deps\usr\bin. After that it should all work (at least it does for me!).

3 Likes

Thank you! It finally worked. I added that folder to my user Path in environment variables, and it worked!

julia> using LinearAlgebra
julia>  BLAS.vendor()
:mkl

Hi, catching up on stuff now.

The current state of things is that you can ] add https://github.com/arnavs/ArpackMKL.jl on Linux machines no problem.

OS support is provided by https://github.com/arnavs/ArpackMKLBuilder, which provides the MKL-built Arpack binaries for use in the main (“consumer-facing”) package.

On Windows, I was having trouble with the .dll setup vs the .dylib, .so which is used in Linux. This whole thing is also above my pay grade technically, which makes it hard to debug.

4 Likes

A couple of days ago, I requested a pull to their readme tutorial adding those commands we found out from this thread. However, they have not accepted that yet.

1 Like

I gave this a go on Windows, and here’s the step of the build script which failed. I’m looking at the ArpackBuilder script though, and there isn’t any Windows-specific fudging. So this should be possible in principle.

The specific error looks like it’s using -rdynamic, which should be replaced with --export-dynamic in later gcc versions? Not sure. Perhaps @andreasnoack can advise.

sandbox:$WORKSPACE/srcdir/arpack-ng/build# cmake ..  -DINTERFACE64=1 -DBUILD_SHARED_LIBS=ON -DBLAS_LIBRARIES="-L$MKLROOT/lib -lmkl_rt" -DLAPACK_LIBRARIES="-L$MKLROOT/lib -lmkl_rt" -DCMAKE_EXE_LINKER_FLAGS="${CMAKE_EXE_LINKER_FLAGS}" -DCMAKE_Fortran_FLAGS="${FFLAGS}"
-- The C compiler identification is GNU 4.8.5
-- The Fortran compiler identification is GNU 4.8.5
-- Check for working C compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc
-- Check for working C compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc -- broken
CMake Error at /usr/share/cmake/Modules/CMakeTestCCompiler.cmake:52 (message):
  The C compiler

    "/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /workspace/srcdir/arpack-ng/build/CMakeFiles/CMakeTmp
    
    Run Build Command:"/usr/bin/make" "cmTC_d442d/fast"
    /usr/bin/make -f CMakeFiles/cmTC_d442d.dir/build.make CMakeFiles/cmTC_d442d.dir/build
    make[1]: Entering directory '/workspace/srcdir/arpack-ng/build/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_d442d.dir/testCCompiler.c.o
    /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc    -o CMakeFiles/cmTC_d442d.dir/testCCompiler.c.o   -c /workspace/srcdir/arpack-ng/build/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_d442d
    /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_d442d.dir/link.txt --verbose=1
    /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc      -rdynamic CMakeFiles/cmTC_d442d.dir/testCCompiler.c.o  -o cmTC_d442d 
    x86_64-w64-mingw32-gcc: error: unrecognized command line option '-rdynamic'
    make[1]: *** [CMakeFiles/cmTC_d442d.dir/build.make:87: cmTC_d442d] Error 1
    make[1]: Leaving directory '/workspace/srcdir/arpack-ng/build/CMakeFiles/CMakeTmp'
    make: *** [Makefile:126: cmTC_d442d/fast] Error 2
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:7 (project)
1 Like

@dawbarton. Thanks I have been able to build Julia with MKL. I am using Julia 1.1.1 on windows 10.
My problem is that Julia startup take almost 10s. Without MKL, it is just fraction of a second or two.

Is there anything I need to do? I have noticed that first julia startup after MKL build takes around 4s but subsequent startup become slower. Can not figure out the reason.

The problem is that PackageCompiler rebuilds the Julia system image without including the relevant REPL statements. If I understand the code correctly, the file https://github.com/JuliaLang/julia/blob/master/contrib/generate_precompile.jl generates all the necessary precompile statements for the REPL and it’s just a case of including it as part of the PackageCompiler rebuild. I did try doing this on Windows but I wasn’t successful (some problems with Windows not having pseudo-TTYs if I remember correctly) and I didn’t dig any further since using the MKL isn’t high on my priority list.

1 Like

Why not just build Julia and link it against MKL (instead of using MKL.jl)? Building Julia is really easy. Even on Windows it’s not that difficult. See here for instructions.

1 Like

Thanks

That’s not true! It is not easy to build. See the spoiler in the OP for my original question regarding problems in building the Julia with MKL.

For now, MKL.jl has solved my issue. Julia runs fast enough. It is not 10 Seconds! Maybe 1-2 seconds.
The problem is that the repository is inactive. Neither they merge pull requests (see my PR) nor they respond to the issues (see my issue). That happens when 8 people manage 50 repository. Lack of people!

I’m sorry but your OP confuses me - it is a mixture of MKL.jl and building Julia from scratch with non-permalinks.

What issues are you facing with the instructions I linked? To be concrete, these windows instructions.

Personally, I always found MKL.jl pretty much useless because of its limitations - and somewhat hacky.