Error using Julia on an HPC cluster: Juliaup configuration is locked by another process, waiting for it to unlock

Specific Domains Julia at Scale
1

I have been using Julia on an HPC slurm cluster with no problem. However, I was trying to use the MPI.jl package to parallelize my code. The cluster has intel MPI that I have been trying to use without success. I added the MPI.jl package using Pkg.add(“MPI”). My job submission script looks as follows:

#SBATCH --partition short
#SBATCH --N 1 
...
module load intel/MPI
mpirun -n 100 julia script.jl

However, this does not work and I receive an error message saying:

Juliaup configuration is locked by another process, waiting for it to unlock.

The previous error message is repeated around 100 times in the error file “stderr”, and since I have 100 MPI ranks, this suggests that each MPI rank is waiting for the configuration file to get unlocked.

Now, Julia itself is not working anymore and I keep receiving the same message. I understand that my unsuccessful attempt to use MPI led to some kind of lock on a certain configuration file. However, I have cancelled the MPI jobs I have been trying to do, logged off and on again, but the issue is still there. I have two questions:

  1. How to properly use MPI.jl on a Slurm cluster? After checking online, it looks like I need to add another package “MPIPreferences.jl” to make this work? The documentation does not seem very clear to me.
  2. How to resolve this issue of locked file?
2

What’s unclear about it? If you want to use a system MPI (the Intel MPI provided by module load intel/MPI on your cluster) you have to do

using MPIPreferences
MPIPreferences.use_system_binary()

as is described in the docs. Afterwards. the mpirun call should work fine.

Having said that, you might want to give the MPI binaries (JLLs) that we ship automatically a try. For this, drop the module loading part and use mpiexecjl instead of mpirun. Note that you have to do

using MPI
MPI.install_mpiexecjl()

and have to put ~/.julia/bin on $PATH for it to be available. And if you did the MPIPreferences.use_system_binary() part above, you have to undo it with MPIPreferences.use_jll_binary().

3

This seems to be a juliaup issue ("Juliaup configuration is locked by another process, waiting for it to unlock." · Issue #598 · JuliaLang/juliaup · GitHub), and the solution might be to run

juliaup config versionsdbupdateinterval 0
juliaup config backgroundselfupdateinterval 0

to disable checks for updates. Alternately, you may launch julia by specifying the binary path directly instead of using juliaup.

1 Like