Although I’m a happy nix user indeed, I wouldn’t call depending on nix in CI jobs nirvana per se. Note that CI jobs are very important for most Julia developers. The whole CI ecosystem is built around stateful operating systems. I have used nix in CI and even created a GitHub Action but can’t say that it has been a pleasant experience.
Like I said, unlike other languages, Julia has a solid package manager, so I don’t see the problem of using it.
Good point.
The general product category of CI is mostly not great, so I can’t disagree on that point. Fwiw there are a bunch of CI services specifically built around being good for Nix. Drone, Hercules, etc.
Increasing Nix immersion is one of those cultural-technical experiences that can promote a certain way of thinking – that I happen to like, namely using stateless systems everywhere.
@ninjin is the new julia_16-bin expected to work with nix-shell? I’ve been trying it with nix-shell -p julia_16-bin although I can start the julia repl and get to the pkg prompt and install Plots and GR but when I run using Plots; plot(rand(10)) I get:
env: ‘/home/larry/.julia/artifacts/716c009a184899dcd4e14fc566e3554d3e92f796/bin/gksqt’: No such file or directory
Followed by a bunch of messages like
GKS: GKS not in proper state. GKS must be either in the state WSOP or WSAC in routine ACTIVATE_WS
Is there a different way I should use this new package?
No, what you are experiencing is not due how you use the package but rather a standard NixOS issue: The gksqt binary needs to be patched with the correct interpreter. We should do this automatically by patching Pkg in the standard library, but I have not found the cycles to do this cleanly â although it should be fairly darn easy. But you can do it with a dirty hack for now:
chmod +w ~/.julia/artifacts/716c009a184899dcd4e14fc566e3554d3e92f796/bin/gksqt; nix-shell -p patchelf stdenv --command "patchelf --set-interpreter \"\$(cat \${NIX_CC}/nix-support/dynamic-linker)\" ~/.julia/artifacts/716c009a184899dcd4e14fc566e3554d3e92f796/bin/gksqt"
This messes up the artifact checksum, but it gets you up and working.
Ah ok, that’s gotten me further I can now plot simple things. Trying to run the Lorenz attractor demo from the Plots homepage hits a similar issue with ffmpeg, but now that I know what’s going on I was able to figure my way through it.
Thanks for the help and the work on Julia in NixPkgs, I’m looking forward to a more complete Julia system on Nix!
Does anybody have GLMakie working?
I still can’t get plots.
NIX_PATH=nixpkgs=http://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz nix-shell -p julia_16-bin
pkg> add GRUtils
julia> ENV["GRDIR"] = ""
pkg> build GR
julia> using GRUtils
julia> x = LinRange(0, 10, 500)
julia> y = sin.(x.^2) .* exp.(-x)
julia> GR.plot(x, y)
yields this:
ERROR: could not load library “libGR.so”
/nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libc.so.6: version `GLIBC_2.33’ not found (required by /home/dkahlenberg/.julia/packages/GR/4DHy8/src/…/deps/gr/lib/libGR.so)
To reply to myself GR build problems: "installation is incomplete" - #26 by mkitti had the answer (somewhat hidden in the comment text) it was to use the explicit version providing BinaryBuilder as in:
pkg> add GR@0.57
Just popping in to say if you want to help add support for Julia on https://Repl.it with Nix, please feel free to chime in: Bump Julia to 1.5.3 by logankilpatrick · Pull Request #222 · replit/polygott · GitHub
Hello,
Here there is a nix code that is working for me with Julia 1.5 to produce plots. I think that a similar version has been shared before in this thread, but given that some users are asking for something that can produce plots, here it is one (for NixOS 20.09)
In my configuration.nix, it is included a call to a package where I define the julia 1.5 compilation with the needed libraries for plotting.
{ config, pkgs, lib, ... }:
let
myjulia = pkgs.callPackage ./julia_new15 {};
in
{ ... }
the usual NixOS configuration can be inside the {…} but adding to environment.systemPackages the package above defined myjulia.
… and here is the file julia_new15. nix:
{ stdenv, fetchurl, fetchzip, fetchFromGitHub
# build tools
, gfortran, m4, makeWrapper, patchelf, perl, which, python2, python3, R, gmt
, cmake, netcdf, netcdffortran, hdf5-fortran
# libjulia dependencies
, libunwind, readline, utf8proc, zlib
# standard library dependencies
, curl, fftwSinglePrec, fftw, libgit2, mpfr, openlibm, openspecfun, pcre2
# linear algebra
, blas, lapack, arpack
}:
assert (!blas.isILP64) && (!lapack.isILP64);
with stdenv.lib;
let
majorVersion = "1";
minorVersion = "5";
maintenanceVersion = "3";
src_sha256 = "sha256:0jds8lrhk4hfdv7dg5p2ibzin9ivga7wrx7zwcmz6dqp3x792n1i";
version = "${majorVersion}.${minorVersion}.${maintenanceVersion}";
in
stdenv.mkDerivation rec {
pname = "julia";
inherit version;
src = fetchzip {
url = "https://github.com/JuliaLang/julia/releases/download/v${version}/julia-${version}-full.tar.gz";
sha256 = src_sha256;
};
patches = [
./use-system-utf8proc-julia-1.3.patch
# Julia recompiles a precompiled file if the mtime stored *in* the
# .ji file differs from the mtime of the .ji file. This
# doesn't work in Nix because Nix changes the mtime of files in
# the Nix store to 1. So patch Julia to accept mtimes of 1.
./allow_nix_mtime.patch
];
postPatch = ''
patchShebangs . contrib
for i in backtrace cmdlineargs; do
mv test/$i.jl{,.off}
touch test/$i.jl
done
rm stdlib/Sockets/test/runtests.jl && touch stdlib/Sockets/test/runtests.jl
rm stdlib/Distributed/test/runtests.jl && touch stdlib/Distributed/test/runtests.jl
# LibGit2 fails with a weird error, so we skip it as well now
rm stdlib/LibGit2/test/runtests.jl && touch stdlib/LibGit2/test/runtests.jl
sed -e 's/Invalid Content-Type:/invalid Content-Type:/g' -i ./stdlib/LibGit2/test/libgit2.jl
sed -e 's/Failed to resolve /failed to resolve /g' -i ./stdlib/LibGit2/test/libgit2.jl
'';
dontUseCmakeConfigure = true;
buildInputs = [
arpack fftw fftwSinglePrec libgit2 libunwind mpfr
pcre2.dev blas lapack openlibm openspecfun readline utf8proc
zlib python3 python2 R
];
nativeBuildInputs = [ curl gfortran m4 makeWrapper patchelf perl python2 python3 which cmake R gmt netcdf netcdffortran hdf5-fortran ];
makeFlags =
let
arch = head (splitString "-" stdenv.system);
march = {
x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64";
i686 = "pentium4";
aarch64 = "armv8-a";
}.${arch}
or (throw "unsupported architecture: ${arch}");
# Julia requires Pentium 4 (SSE2) or better
cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch}
or (throw "unsupported architecture: ${arch}");
# Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM
# https://github.com/JuliaLang/julia/tree/master/deps/patches
in [
"ARCH=${arch}"
"MARCH=${march}"
"JULIA_CPU_TARGET=${cpuTarget}"
"PREFIX=$(out)"
"prefix=$(out)"
"SHELL=${stdenv.shell}"
"USE_SYSTEM_BLAS=1"
"USE_BLAS64=${if blas.isILP64 then "1" else "0"}"
"USE_SYSTEM_LAPACK=1"
"USE_SYSTEM_ARPACK=1"
"USE_SYSTEM_FFTW=1"
"USE_SYSTEM_GMP=0"
"USE_SYSTEM_LIBGIT2=1"
"USE_SYSTEM_LIBUNWIND=1"
"USE_SYSTEM_MPFR=1"
"USE_SYSTEM_OPENLIBM=1"
"USE_SYSTEM_OPENSPECFUN=1"
"USE_SYSTEM_PATCHELF=1"
"USE_SYSTEM_PCRE=1"
"PCRE_CONFIG=${pcre2.dev}/bin/pcre2-config"
"PCRE_INCL_PATH=${pcre2.dev}/include/pcre2.h"
"USE_SYSTEM_READLINE=1"
"USE_SYSTEM_UTF8PROC=1"
"USE_SYSTEM_ZLIB=1"
"USE_BINARYBUILDER=0"
];
LD_LIBRARY_PATH = makeLibraryPath [
arpack fftw fftwSinglePrec libgit2 mpfr blas openlibm
openspecfun pcre2 lapack python3 python3 R
];
enableParallelBuilding = true;
# Julia's tests require read/write access to $HOME
preCheck = ''
export HOME="$NIX_BUILD_TOP"
'';
preBuild = ''
sed -e '/^install:/s@[^ ]*/doc/[^ ]*@@' -i Makefile
sed -e '/[$](DESTDIR)[$](docdir)/d' -i Makefile
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}
'';
postInstall = ''
# Symlink shared libraries from LD_LIBRARY_PATH into lib/julia,
# as using a wrapper with LD_LIBRARY_PATH causes segmentation
# faults when program returns an error:
# $ julia -e 'throw(Error())'
find $(echo $LD_LIBRARY_PATH | sed 's|:| |g') -maxdepth 1 -name '*.${if stdenv.isDarwin then "dylib" else "so"}*' | while read lib; do
if [[ ! -e $out/lib/julia/$(basename $lib) ]]; then
ln -sv $lib $out/lib/julia/$(basename $lib)
fi
done
'';
passthru = {
inherit majorVersion minorVersion maintenanceVersion;
site = "share/julia/site/v${majorVersion}.${minorVersion}";
};
meta = {
description = "High-level performance-oriented dynamical language for technical computing";
homepage = "https://julialang.org/";
license = stdenv.lib.licenses.mit;
maintainers = with stdenv.lib.maintainers; [ raskin rob garrison ];
platforms = [ "i686-linux" "x86_64-linux" "x86_64-darwin" "aarch64-linux" ];
broken = stdenv.isi686;
};
}
Do you have any more thoughts on patching Pkg to make this work? I’ve been looking at it a tiny bit, but I’m pretty new to Julia so I’m not sure if I’m on the right track.
I saw where you suggested we might insert the call to patchElf, but do you think maybe here (i.e. after the hash has been calculated, but before the write bit is cleared) is a better option?
Do you have a sense for how “careful” we need to be with patchElf? Can we indiscriminately attempt to patchElf everything that comes in an artifact’s bin directory, or do we need to somehow test each one before patching in the nix dynamic linker?
I’m not familiar with the Julia Pkg system, are there cases we need to lookout for where stuff under lib also needs to be patchElf’d or are those standalone from Pkg already?
I did something… I’m not sure I’m proud of it, but it’s at least a bit more convenient to get a new package installed. Here’s my shell.nix:
{pkgs ? import<nixpkgs>{} }:
with pkgs;
let
fixJuliaPkgs = writeScriptBin "fixJuliaPkgs" ''
#!/usr/bin/env bash
PKG_DIR=~/.julia/
for ARTIFACT in $(find $PKG_DIR/artifacts/*/bin)
do
chmod +w $ARTIFACT
${patchelf}/bin/patchelf \
$ARTIFACT \
--set-interpreter \
"$(cat $NIX_CC/nix-support/dynamic-linker)"
chmod -w $ARTIFACT
done
'';
in
mkShell {
buildInputs = [
fixJuliaPkgs
julia-stable-bin
];
}
Basically, it just introduces the fixJuliaPkgs script which attempts to patchelf everything in all the artifact bin/ directories. I manually call it after adding a package with Pkg. So far, this seems to work OK because patchelf is smart enough to only set the dynamic linker on files which are actually elf binaries, and which have a linker to set. It’s not elegant, but it’s a stop-gap for me until I find something better or nixpkgs patches Pkg to automatically patchelf on download.
I’ve also got this abomination on GitHub. Feedback welcome!
hope someone fix it 
Hi all, I realize this is an old thread but I’ve been working on a more serious way to build Julia environments on NixOS. You can find it here:
Feedback is welcome! If enough interested Julia users leave a thumbs up, maybe it will help overcome Nixpkgs maintainers’ dislike for import-from-derivation 
This sounds great - I will be trying this out over the next month or so on darwin (macos M2)