Cannot succeed in compiling julia

installation

#1

Hello
It do not succeed in installing Julia (v. 0.7.0-full, downloaded as a source tgz). I tried two gnu/linux based machines: redhat 7, linux Mint workstation, gcc 5.4.0, clang 3.8.0. Always the same problem: everything OK, but at the end of the compilation I get the message:

jl_write_compiler_output at /home/manu/TMP/JULIA/julia-0.7.0-full/src/precompile.c:68
jl_atexit_hook at /home/manu/TMP/JULIA/julia-0.7.0-full/src/init.c:233
main at /home/manu/TMP/JULIA/julia-0.7.0-full/ui/repl.c:234
__libc_start_main at /build/glibc-Cl5G7W/glibc-2.23/csu/…/csu/libc-start.c:291
_start at /home/manu/TMP/JULIA/julia-0.7.0-full/usr/bin/julia (unknown line)
Generating precompile statements… 774 generated in 50.124638 seconds
*** This error is usually fixed by running make clean. If the error persists, try make cleanall. ***
Makefile:216: recipe for target ‘/home/manu/TMP/JULIA/julia-0.7.0-full/usr/lib/julia/sys-o.a’ failed
make[1]: *** [/home/manu/TMP/JULIA/julia-0.7.0-full/usr/lib/julia/sys-o.a] Error 1
Makefile:78: recipe for target ‘julia-sysimg-release’ failed
make: *** [julia-sysimg-release] Error 2

Then I try make clean, make cleanall then make again: go back to the same point. So at the end the file sys.so is missing, and julia does not start.

Am i doing anything wrong ?

Thanks for your help !
Emmanuel


#2

It looks like the error is from the precompiliation part, which is rather memory demanding and could fail if you don’t have enough memory. You can try to set JULIA_PRECOMPILE=0 in Make.user and try again.


#3

Thanks for your help. However, the result is unchanged (I did not compile from scratch, I just tried make -j 4 again). Here is my Make.user:

cat Make.user
BUILD_LLVM_CLANG := 1
USECLANG := 1
JULIA_PRECOMPILE := 0

The versions are:

clang --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

cat /etc/*-release
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=18.3
DISTRIB_CODENAME=sylvia

uname -a
Linux hplap1 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

free
total used free shared buff/cache available
Mem: 8046904 2550380 636164 393432 4860360 4734488
Swap: 8261628 655964 7605664

Sincerely,
Emmanuel


#4

@manu May I ask why 0.7 The latest version is 1.0.2
OF course you may have a specific need for 0.7 and the deprecation warnings it has. But surely on a new project you should go with 1.0.x ??


#5

While asking stupid questions: why can’t you use ready binaries from download page?


#6

Hello

No your questions are not stupid. Here is the ultimate goal: installing julia on our supercomputer (cf. https://www.calmip.univ-toulouse.fr/) and make it available to our users through a gnu module (http://modules.sourceforge.net/). We recompile everything to get best performance (using intel mkl etc).

We install the 0.7.0 release because some of our users need compatibility with “old” work. However we’ll install also the 1.0.2 release, for new projects. Thanks to the Environment modules program it is easy to install several versions of softwares.

However, this is not an easy task compiling julia, and to start I am trying to compile Julia on my gnu/linux mint based laptop, because I thought it would be easier doing this on a quite standard machine. When I’ll succeed in this install (including tests), I’ll try to do the same on the supercomputer.

Sincerely,
Emmanuel


#7

Perhaps try to capture the complete log and post that.


#8

It would be cool, if you could run benchmark comparison of ready binaries and compiled ones on a cluster, after success in building and installing. This way others, me included, could make a conclusion if it is worth the effort.


#9

@manu Fantastic! I work in HPC myself. On my own laptop I have Modules and a couple of different versions of Julia.
There should be an Easybuild or a Spack recipe for Julia . Update - I am not familiar with Spack but there is a recipe for Julia. I know Easybuild better.
As @Tero_Frondelius says - please if you can share with us any comparisons between ready built binaries and optimised ones.


#10

Well

Using the release 1.0.2 the compilation succeeds on my laptop, make testall succeeds too.
So I tried compiling 1.0.2 on the supercomputer. Using gcc 4.8.5 and the release 18.2 of intel MKL: the compilation succeeds, but the tests fail with the message:

ccall: could not find function __ldexp_expf in library libimf

I am not the first: see Use_intel_libm = 1
So, is there any solution for using Julia 1.0.2 with the mkl or is the issue still open ?

Sincerely,
Emmanuel


#11

As I see - it is changed to


#12

hoho ! Thanks !
So here is the new Make.user:

prefix = /usr/local/julia/1.0.2
USEIFC = 1
USE_INTEL_MKL = 1

Compilation OK, but 4 tests failed:

Worker 4 failed running test cmdlineargs:
Some tests did not pass: 202 passed, 4 failed, 0 errored, 0 broken.cmdlineargs: Test Failed at /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:422
Expression: success(#= /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:422 =# @cmd("$exename -e “exit(0)”"))
Stacktrace:
[1] record(::Test.DefaultTestSet, ::Test.Fail) at /tmpdir/sysinst/julia/julia-1.0.2-full/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:745
[2] (::getfield(Main, Symbol("##42#48")))() at /tmpdir/sysinst/julia/julia-1.0.2-full/test/runtests.jl:237
[3] cd(::getfield(Main, Symbol("##42#48")), ::String) at ./file.jl:96
[4] top-level scope at none:0
[5] include at ./boot.jl:317 [inlined]
[6] include_relative(::Module, ::String) at ./loading.jl:1044
[7] include(::Module, ::String) at ./sysimg.jl:29
[8] exec_options(::Base.JLOptions) at ./client.jl:231
[9] _start() at ./client.jl:425
cmdlineargs: Test Failed at /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425
Expression: success(#= /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425 =# @cmd("$exename -e “exit(!(Base.load_path() == ))”"))
Stacktrace:
[1] record(::Test.DefaultTestSet, ::Test.Fail) at /tmpdir/sysinst/julia/julia-1.0.2-full/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:745
[2] (::getfield(Main, Symbol("##42#48")))() at /tmpdir/sysinst/julia/julia-1.0.2-full/test/runtests.jl:237
[3] cd(::getfield(Main, Symbol("##42#48")), ::String) at ./file.jl:96
[4] top-level scope at none:0
[5] include at ./boot.jl:317 [inlined]
[6] include_relative(::Module, ::String) at ./loading.jl:1044
[7] include(::Module, ::String) at ./sysimg.jl:29
[8] exec_options(::Base.JLOptions) at ./client.jl:231
[9] _start() at ./client.jl:425
cmdlineargs: Test Failed at /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425
Expression: success(#= /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425 =# @cmd("$exename -e “exit(!(Base.load_path() == ))”"))
Stacktrace:
[1] record(::Test.DefaultTestSet, ::Test.Fail) at /tmpdir/sysinst/julia/julia-1.0.2-full/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:745
[2] (::getfield(Main, Symbol("##42#48")))() at /tmpdir/sysinst/julia/julia-1.0.2-full/test/runtests.jl:237
[3] cd(::getfield(Main, Symbol("##42#48")), ::String) at ./file.jl:96
[4] top-level scope at none:0
[5] include at ./boot.jl:317 [inlined]
[6] include_relative(::Module, ::String) at ./loading.jl:1044
[7] include(::Module, ::String) at ./sysimg.jl:29
[8] exec_options(::Base.JLOptions) at ./client.jl:231
[9] _start() at ./client.jl:425
cmdlineargs: Test Failed at /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425
Expression: success(#= /tmpdir/sysinst/julia/julia-1.0.2-full/test/cmdlineargs.jl:425 =# @cmd("$exename -e “exit(!(Base.load_path() == ))”"))
Stacktrace:
[1] record(::Test.DefaultTestSet, ::Test.Fail) at /tmpdir/sysinst/julia/julia-1.0.2-full/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:745
[2] (::getfield(Main, Symbol("##42#48")))() at /tmpdir/sysinst/julia/julia-1.0.2-full/test/runtests.jl:237
[3] cd(::getfield(Main, Symbol("##42#48")), ::String) at ./file.jl:96
[4] top-level scope at none:0
[5] include at ./boot.jl:317 [inlined]
[6] include_relative(::Module, ::String) at ./loading.jl:1044
[7] include(::Module, ::String) at ./sysimg.jl:29
[8] exec_options(::Base.JLOptions) at ./client.jl:231
[9] _start() at ./client.jl:425

Looks like this issue: https://github.com/JuliaLang/julia/issues/27940

Sincerely,
Emmanuel


#13

My Make.user simply says:

MARCH = native
USE_INTEL_MKL = 1
USE_INTEL_LIBM = 1

also, be sure to source Intel’s script compilervars.sh. I’ve never had an issue building from source while using anything from gcc-5.4 through gcc 8.2.1.
Watch out though, that ARPACK will not work when you build with MKL, and many packages depend on it, including Distributions and LightGraphs.

EDIT:
I’m trying make testall now. I never had a problem before.
I can confirm that I have the same test failures.


#14

Thank you Elrod for testing again

So now it is clear: no way installing Julia 1.0.2 (with successful tests) with the MKL. I installed a release “nomkl”.

About benchmarks, did you refer to the micro-benchmarks https://github.com/JuliaLang/Microbenchmarks ?

Sincerely,
Emmanuel