Building Julia for Nix

General Usage
1

Nix is this nice tool for reproducible builds on Linux and MacOS that I’ve been trying to learn.

Currently the main nixpkgs repository only builds Julia on Linux[1] (both x86_64 and aarch64), so I’m trying to add support for MacOS. Here is the Julia build listed in the tree: nixpkgs/pkgs/development/compilers/julia at dd443d2d10ff7f7f7451f5bd04268f21daceb74d · NixOS/nixpkgs · GitHub

I’ve got things like 99% of the way working with just these changes: Comparing NixOS:master...MilesCranmer:master · NixOS/nixpkgs · GitHub. There is only a single error now. This error is as follows:

libtool: compile:  clang -mmacosx-version-min=11.0 -DHAVE_CONFIG_H -I. -I/private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1 -D__GMP_WITHIN_GMP -O2 -pedantic -march=armv8-a -c /private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1/tal-reent.c  -fno-common -DPIC -o .libs/tal-reent.o
[ 19%] Building CXX object lib/Remarks/CMakeFiles/LLVMRemarks.dir/RemarkLinker.cpp.o
/private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1/tal-reent.c:66:7: error: call to undeclared library function 'abort' with type 'void (void) __attribute__((noreturn))'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
      __GMP_ALLOC_OVERFLOW_FUNC ();
      ^
/private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1/gmp-impl.h:737:5: note: expanded from macro '__GMP_ALLOC_OVERFLOW_FUNC'
    abort ();                                                    \
    ^
/private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1/tal-reent.c:66:7: note: include the header <stdlib.h> or explicitly provide a declaration for 'abort'
/private/tmp/nix-build-julia-1.10.4.drv-1/julia-1.10.4/deps/srccache/gmp-6.2.1/gmp-impl.h:737:5: note: expanded from macro '__GMP_ALLOC_OVERFLOW_FUNC'
    abort ();                                                    \
    ^
1 error generated.
[ 20%] Building CXX object lib/MC/CMakeFiles/LLVMMC.dir/SPIRVObjectWriter.cpp.o

Does anybody know how to fix this? Is it from some patched version of GMP that Julia is using but isn’t available in Nix or something? (I’m assuming not, because the dependencies are listed in srccache which I’m guessing are the ones Julia packages)

  1. Though note there is a julia-bin which simply re-packages the binaries. ↩︎

1 Like
2

Got a bit farther with

  NIX_CFLAGS_COMPILE = [] ++ lib.optionals stdenv.isDarwin [
    "-Wno-error=implicit-function-declaration"
  ];

Not sure what is going on with that GMP dependency.

3

The Julia build of GMP is described here:

4

I think those are the ones being used during the nix build? It just executes make on the Julia source tree. The only explicit dependencies to the nix build are:

{ lib
, stdenv
, fetchurl
, which
, python3
, gfortran
, cmake
, perl
, gnum4
, openssl
, libxml2
, unzip
1 Like
5

Well, it looks like Miles’s branch is not using the JLL: nixpkgs/pkgs/development/compilers/julia/generic.nix at 42c55c14c40b767cbbe6e7d636766ff0dd8f1f58 · MilesCranmer/nixpkgs · GitHub

Because of this line (line 56)

"USE_BINARYBUILDER=0"
6

Which would mean that Miles’s build is doing a source-build of GMP (with our GMP patches applied).

Looks like we only carry two patches for GMP:

  1. julia/deps/patches/gmp-alloc_overflow.patch at master · JuliaLang/julia · GitHub
  2. julia/deps/patches/gmp-exception.patch at master · JuliaLang/julia · GitHub
7

@MilesCranmer Can you reproduce the same error just by doing make USE_BINARYBUILDER=0 on your macOS machine (not using Nix)?

8

Thanks, indeed it looks like it’s just building GMP from source. Though there is the “julia-bin” version which is just pre-packaged Julia binaries.

Sadly I’ve never had much luck building Julia from source on my machine with the standard workflow. I think I’ve tried building it ~20 times and only gotten through on ~3 tries – there seems to be some weird interaction with my environment variables or system clang or brew that always breaks something, and by that point I usually move to Linux instead.

Actually the reason I’m learning Nix is precisely so I can build Julia in a repeatable way on my mac. Once this works it should always work which is nice. (Kind of like Docker, but without needing to run in a virtual machine)

9

Okay seems to get a bit farther with that -Wno-error=implicit-function-declaration. Now stuck at

configure: error: couldn't find libraries for gethostbyname()
10

Could it be from the Julia makefile forcing xcode on any Darwin-like system?

The Julia Make.inc has xcrun hard-coded:

# Note: we're passing *FLAGS here computed based on your system compiler to
# clang. If that causes you problems, you might want to build and/or run
# specific clang-sa-* files with clang explicitly selected:
#   make CC=~+/../usr/tools/clang CXX=~+/../usr/tools/clang USECLANG=1 analyzegc
#   make USECLANG=1 clang-sa-*
CLANGSA_FLAGS :=
CLANGSA_CXXFLAGS :=
ifeq ($(OS), Darwin) # on new XCode, the files are hidden
 CLANGSA_FLAGS += -isysroot $(shell xcrun --show-sdk-path -sdk macosx)
endif

However, the nix docs say:

Some packages assume xcode is available and use xcrun to resolve build tools like clang, etc. This causes errors like xcode-select: error: no developer tools were found at '/Applications/Xcode.app' while the build doesn’t actually depend on xcode.

stdenv.mkDerivation {
  name = "libfoo-1.2.3";
  # ...
  prePatch = ''
    substituteInPlace Makefile \
        --replace-fail '/usr/bin/xcrun clang' clang
  '';
}

The package xcbuild can be used to build projects that really depend on Xcode. However, this replacement is not 100% compatible with Xcode and can occasionally cause issues.

11

@sjkelly have you been using nix on your mac?

1 Like
12

Got a bit further with some hacks:

, xcbuild

as a dependency (nixpkg that acts like xcode) and

makeFlags = [
    "USE_BINARYBUILDER=1"
]

I think Nix people like to build everything from source, but since the only other option to USE_BINARYBUILDER=1 is to use the entire prepackaged binary (i.e., julia-bin), I think it’s a decent tradeoff for having a source-compiled Julia on mac.

and

NIX_CFLAGS_COMPILE = [] ++ lib.optionals stdenv.isDarwin [
  "-Wno-error=implicit-function-declaration"
  "-Wno-error=elaborated-enum-base"
];

I’m not sure what those errors are for but the dependency didn’t seem too important?

The build error is now just this:

lld: error: malformed minimum version:

while precompiling SparseArrays.

Full error message: 
    JULIA stdlib/SparseArrays.release.image
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
ERROR: ERROR: ERROR: failed process: failed process: failed process: Process(Process(Process(setenv(setenv(setenv(...
...
make[1]: *** [pkgimage.mk:92: stdlib/libLLVM_jll.release.image] Error 1
make[1]: *** [pkgimage.mk:89: stdlib/PCRE2_jll.release.image] Error 1
lld: error: malformed minimum version: 
lld: error: malformed minimum version: 
ERROR: failed process: Process(setenv(...

...
mp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10", "NIX_ENFORCE_PURITY=1", "propagatedNativeBuildInputmake[1]: *** [pkgimage.mk:79: stdlib/Profile.release.image] Error 1
s=", "ZERO_AR_DATE=1", "stdenv=/nix/store/q2sr9a4likgf0c0ib3zxxy63vr0dr0w9-stdenv-darwin", "NIX_HARDENING_ENABLE=bindnow format fortify fortify3 pic relro stackprotector strictoverflow", "MAKE_TERMOUT=/dev/ttys004", "JULIA_CPU_TARGET=generic;cortex-a57;thunderx2t99;carmel,clone_all;apple-m1,base(3);neoverse-512tvb,base(3)", "USE_BINARYBUILDER=1", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:565 [inlined]
  [2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
    @ Base ./process.jl:480
  [3] run
    @ ./process.jl:477 [inlined]
  [4] link_image (repeats 2 times)
    @ ./linking.jl:166 [inlined]
  [5] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2379
  [6] compilecache
    @ ./loading.jl:2340 [inlined]
  [7] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1974
  [8] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
  [9] #mkpidlock#6
    @ /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
 [10] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
 [11] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
 [12] invokelatest
    @ ./essentials.jl:889 [inlined]
 [13] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2983
 [14] maybe_cachefile_lock
    @ ./loading.jl:2980 [inlined]
 [15] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1970
 [16] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [17] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [18] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [19] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [20] macro expansion
    @ ./loading.jl:1790 [inlined]
 [21] macro expansion
    @ ./lock.jl:267 [inlined]
 [22] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [23] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [24] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [25] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [26] include(mod::Module, _path::String)
    @ Base ./Base.jl:495
 [27] include(x::String)
    @ SparseArrays /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/SparseArrays/src/SparseArrays.jl:6
 [28] top-level scope
    @ /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/SparseArrays/src/SparseArrays.jl:77
 [29] include
    @ ./Base.jl:495 [inlined]
 [30] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2222
 [31] top-level scope
    @ stdin:3
in expression starting at /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/SparseArrays/src/solvers/LibSuiteSparse.jl:1
in expression starting at /private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/stdlib/v1.10/SparseArrays/src/SparseArrays.jl:3
in expression starting at stdin:3
ERROR: Failed to precompile SparseArrays [2f01184e-e22b-5df5-ae63-d93ebab69eaf] to "/private/tmp/nix-build-julia-1.10.4.drv-0/julia-1.10.4/usr/share/julia/compiled/v1.10/SparseArrays/jl_ihJKs9".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2468
 [3] compilecache
   @ ./loading.jl:2340 [inlined]
 [4] compilecache
   @ ./loading.jl:2330 [inlined]
 [5] compilecache(pkg::Base.PkgId)
   @ Base ./loading.jl:2328
 [6] top-level scope
   @ none:1
make[1]: *** [pkgimage.mk:133: stdlib/SparseArrays.release.image] Error 1
make: *** [Makefile:112: stdlibs-cache-release] Error 2
13

That’s likely coming from package images

Which gets this value from julia/base/Makefile at d71441b505473c599a8a3009db92cafd41272197 · JuliaLang/julia · GitHub

1 Like
14

Awesome, thanks! This patch to base/Makefile seems to fix it:

-	@echo "const MACOS_PRODUCT_VERSION = \"$(shell sw_vers -productVersion)\"" >> $@
-	@echo "const MACOS_PLATFORM_VERSION = \"$(shell xcrun --show-sdk-version)\"" >> $@
+	@echo "const MACOS_PRODUCT_VERSION = \"14.1.2\"" >> $@
+	@echo "const MACOS_PLATFORM_VERSION = \"14.4\"" >> $@

I just used my own SDK version; hopefully that won’t break anything on other machines?

It seems like other people often do a similar patch in Nix packages: Code search results · GitHub

Last error… (hopefully)

ERROR: LoadError: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, Your Julia is built with a SSL/TLS engine that libgit2 doesn't know how to configure to use a file or directory of certificate authority roots, but your environment specifies one via the SSL_CERT_FILE variable. If you believe your system's root certificates are safe to use, you can `export JULIA_SSL_CA_ROOTS_PATH=""` in your environment to use those instead.)

I guess I’ll just put in that SSL patch

1 Like
15

Seems to work with those patches. Thanks everyone!

PR here

2 Likes