Hi all,
I have been trying to make Ipopt.jl to work with my Ipopt built against MKL using ICC (which is working without errors, all test passing). Following this topic I modified the file ~/.julia/v0.5/Ipopt/deps/deps.jl to point to my custom Ipopt. Then, I tried to run the test and I got the following error:
MKL FATAL ERROR: Cannot load libmkl_mc3.so or libmkl_def.so.
===================================[ ERROR: Ipopt ]====================================
failed process: Process(
/usr/local/julia/julia-0.5.1/bin/julia -Cx86-64 -J/usr/local/julia/julia-0.5.1/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/users/i/a/iaravena/.julia/v0.5/Ipopt/test/runtests.jl, ProcessExited(1)) [1]
=======================================================================================
ERROR: Ipopt had test errors
in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:749
in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./:0
in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:59
in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./:0
in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
Following other posts for Python with similar errors, I first verified that the directories containing those shared objects were included in LD_LIBRARY_PATH and that other environment variables required for ICC and MKL were set properly. Then I tried to load these shared objects directly to Julia with
cd(“/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/mkl/lib/intel64”) do
Libdl.dlopen(“libmkl_core.so”)
Libdl.dlopen(“libmkl_intel_thread.so”)
end
only to realize that there were undefined symbols in both of them. Those symbols were defined in libiomp5.so, libmkl_core.so and libmkl_intel_thread.so and I was able to load the shared objects causing the MKL FATAL ERROR to Julia using
Libdl.dlopen(“/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/compiler/lib/intel64/libiomp5.so”, Libdl.RTLD_GLOBAL)
cd(“/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/mkl/lib/intel64”) do
Libdl.dlopen(“libmkl_core.so”, Libdl.RTLD_GLOBAL)
Libdl.dlopen(“libmkl_intel_thread.so”, Libdl.RTLD_GLOBAL)
Libdl.dlopen(“libmkl_def.so”, Libdl.RTLD_GLOBAL)
Libdl.dlopen(“libmkl_mc3.so”, Libdl.RTLD_GLOBAL)
end
I included those lines in deps.jl but the original error persisted. Finally, I was able to make everything work by preloading libiomp5.so, libmkl_core.so and libmkl_intel_thread.so when starting Julia, i.e. starting Julia with
LD_PRELOAD=/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_core.so:/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_intel_thread.so:/usr/local/intel/ics_2013.0.028/composer_xe_2013.1.117/compiler/lib/intel64/libiomp5.so julia
however, this is more of a hack than a solution. Is there a proper way to solve the issue of Ipopt not loading these dependencies when called from Julia? Any suggestions are welcome.
Ignacio
PS1: The same problem arose in another newer machine where I also have Ipopt, but with libmkl_avx.so instead of libmkl_mc3.so.
PS2: Both of the machines with the error are in GNU/Linux. I am not sure whether this will happen in other platforms.