# MPI library mismatch erro HDF5, Netcdf with Oceananigans

**URL:** <https://discourse.julialang.org/t/mpi-library-mismatch-erro-hdf5-netcdf-with-oceananigans/131579>\
**Category:** General Usage\
**Tags:** question, hdf5, mpi, netcdf, oceananigans\
**Created:** [August 13, 2025, 1:32pm UTC](https://discourse.julialang.org/t/mpi-library-mismatch-erro-hdf5-netcdf-with-oceananigans/131579 "2025-08-13T13:32:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![farolap](https://avatars.discourse-cdn.com/v4/letter/f/ea666f/32.png) [@farolap](https://discourse.julialang.org/u/farolap)\
**Post date:** [August 13, 2025, 1:32pm UTC](https://discourse.julialang.org/t/mpi-library-mismatch-erro-hdf5-netcdf-with-oceananigans/131579/1 "2025-08-13T13:32:34Z")

</div>

Dear all,

Im having strong issues with a specific package in Julia while trying to run it in MPI. Initially I define the following libraries:

```bash
export LD_LIBRARY_PATH=/sw/spack-levante/netcdf-c-4.8.1-7dq6g2/lib/:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/sw/spack-levante/hdf5-1.12.1-jmeuy3/lib/:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/sw/spack-levante/openmpi-4.1.5-godjja/lib/:$LD_LIBRARY_PATH
export JULIA_HDF5_PATH=/sw/spack-levante/hdf5-1.12.1-jmeuy3/lib/

```

Im not sure if all of them are necessary, but so it was defined. After these definitions i used MPIPreferences to set up the following in Julia:

```julia
using MPIPreferences
MPIPreferences.use_system_binary(mpiexec="srun"; library_names="/sw/spack-levante/openmpi-4.1.5-godjja/lib/libmpi.so")

```

This configuration runs fine if the only library im using is the `MPI`. However, if I import `Oceananigans`, which contains many dependencies, including `JLD2`, `NCDatasets`, etc, then the precompilation of such library lead to a chain of errors. Namely:

```julia
  3 dependencies successfully precompiled in 5 seconds. 23 already precompiled.
Precompiling project...
  ✗ HDF5_jll
  ✗ NetCDF_jll
  ✗ NCDatasets
  ✗ Oceananigans
  ✗ Oceananigans → OceananigansMakieExt
  315 dependencies successfully precompiled in 199 seconds. 41 already precompiled.
  5 dependencies errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 3 direct dependencies failed to precompile:

OceananigansMakieExt 

Failed to precompile OceananigansMakieExt [8b7e02c2-18e1-5ade-af7b-cfb5875075c8] to "/home/m/m300992/.julia/compiled/v1.11/OceananigansMakieExt/jl_spGCGf".
ERROR: LoadError: InitError: could not load library "/home/m/m300992/.julia/artifacts/695bf0397a849eab58cd21bf28f8c4279cfa42ad/lib/libmpi.so"
/home/m/m300992/.julia/artifacts/695bf0397a849eab58cd21bf28f8c4279cfa42ad/lib/libopen-pal.so.80: undefined symbol: pmix_framework_names
Stacktrace:

[...]

during initialization of module OpenMPI_jll
in expression starting at /home/m/m300992/.julia/packages/HDF5_jll/NQMjC/src/wrappers/x86_64-linux-gnu-libgfortran5-cxx11-mpi+openmpi.jl:9
in expression starting at /home/m/m300992/.julia/packages/HDF5_jll/NQMjC/src/HDF5_jll.jl:2
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile HDF5_jll [0234f1f7-429e-5d53-9886-15a909be8d59] to "/home/m/m300992/.julia/compiled/v1.11/HDF5_jll/jl_VuLzHv".
Stacktrace:

[...]

in expression starting at /home/m/m300992/.julia/packages/NetCDF_jll/f16pp/src/wrappers/x86_64-linux-gnu-mpi+openmpi.jl:6
in expression starting at /home/m/m300992/.julia/packages/NetCDF_jll/f16pp/src/NetCDF_jll.jl:2
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile NetCDF_jll [7243133f-43d8-5620-bbf4-c2c921802cf3] to "/home/m/m300992/.julia/compiled/v1.11/NetCDF_jll/jl_5gNMYp".
Stacktrace:

[...]

in expression starting at /home/m/m300992/.julia/packages/NCDatasets/xJGTj/src/NCDatasets.jl:1
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile NCDatasets [85f8d34a-cbdd-5861-8df4-14fed0d494ab] to "/home/m/m300992/.julia/compiled/v1.11/NCDatasets/jl_Z1FxmC".
Stacktrace:

[...]

in expression starting at /work/mh0033/m300992/cci_euler/Oceananigans.jl/src/OutputWriters/netcdf_output_writer.jl:1
in expression starting at /work/mh0033/m300992/cci_euler/Oceananigans.jl/src/OutputWriters/OutputWriters.jl:1
in expression starting at /work/mh0033/m300992/cci_euler/Oceananigans.jl/src/Oceananigans.jl:1
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile Oceananigans [9e8cae18-63c1-5223-a75c-80ca9d6e9a09] to "/home/m/m300992/.julia/compiled/v1.11/Oceananigans/jl_08PDn5".
Stacktrace:

[...]

in expression starting at /work/mh0033/m300992/cci_euler/Oceananigans.jl/ext/OceananigansMakieExt.jl:1
in expression starting at stdin:6
NCDatasets 

Failed to precompile NCDatasets [85f8d34a-cbdd-5861-8df4-14fed0d494ab] to "/home/m/m300992/.julia/compiled/v1.11/NCDatasets/jl_T2tpqM".
ERROR: LoadError: InitError: could not load library "/home/m/m300992/.julia/artifacts/695bf0397a849eab58cd21bf28f8c4279cfa42ad/lib/libmpi.so"
/home/m/m300992/.julia/artifacts/695bf0397a849eab58cd21bf28f8c4279cfa42ad/lib/libopen-pal.so.80: undefined symbol: pmix_framework_names
Stacktrace:

[...]

during initialization of module OpenMPI_jll
in expression starting at /home/m/m300992/.julia/packages/HDF5_jll/NQMjC/src/wrappers/x86_64-linux-gnu-libgfortran5-cxx11-mpi+openmpi.jl:9
in expression starting at /home/m/m300992/.julia/packages/HDF5_jll/NQMjC/src/HDF5_jll.jl:2
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile HDF5_jll [0234f1f7-429e-5d53-9886-15a909be8d59] to "/home/m/m300992/.julia/compiled/v1.11/HDF5_jll/jl_6OxoUC".
Stacktrace:

[...]

in expression starting at /home/m/m300992/.julia/packages/NetCDF_jll/f16pp/src/wrappers/x86_64-linux-gnu-mpi+openmpi.jl:6
in expression starting at /home/m/m300992/.julia/packages/NetCDF_jll/f16pp/src/NetCDF_jll.jl:2
in expression starting at stdin:6
ERROR: LoadError: Failed to precompile NetCDF_jll [7243133f-43d8-5620-bbf4-c2c921802cf3] to "/home/m/m300992/.julia/compiled/v1.11/NetCDF_jll/jl_JgVqvL".
Stacktrace:

[...]

in expression starting at /home/m/m300992/.julia/packages/NCDatasets/xJGTj/src/NCDatasets.jl:1
in expression starting at stdin:6

Oceananigans 

```

Ive tried the help from the vendors of my institute HPC, but they couldn’t help. So, as a last resort, im coming here for help, as i exausted all my knowledge from these hardware specific libraries and all my motivation to keep digging into this problem. I already appreciate everyones attention.

Best,

Fabricio

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 14, 2025, 10:59am UTC](https://discourse.julialang.org/t/mpi-library-mismatch-erro-hdf5-netcdf-with-oceananigans/131579/2 "2025-08-14T10:59:15Z")

</div>

See

> <https://github.com/JuliaParallel/MPI.jl/issues/835#issuecomment-2622909892>
>
> This is what I did on Ookami after I logged on. (Some more details on https://ia…cs-group.slack.com/archives/C016DRQ321M.)
> 
> My setup of the environment looks like this:
> \`\`\`
> export JULIA\_DEPOT\_PATH="~/a64fx/depot"
> module load julia
> module load gcc/13.1.0
> module load slurm
> \# module load mvapich2/arm22/2.3.7
> module load openmpi/gcc8/4.1.2
> \`\`\`
> 
> 1. I cloned the repository FinEtoolsDDParallel.jl from GitHub.
> 2. I activated the environment and instantiated the packages.
>  \`julia -e 'using Pkg; Pkg.activate("."); Pkg.instantiate()'\`
> 3. I ran julia interactively to install mpiexecjl.
>  \`julia --project=.\`
> I also set up the system binary: MPIPreferences.use\_system\_binary().
> 4. I ran srun to get an interactive session.
> \`srun -p short -n 10 --ntasks-per-node=1 --pty bash\`
> 5. I ran the example.
>  cd FinEtoolsDDParallel.jl/examples/
> \`~/a64fx/depot/bin/mpiexecjl -n 4 julia --project=. heat/Poisson2D\_cg\_mpi\_driver.jl\`
>  
> After several minutes, the job was terminated. The error message is below.
> Note well: On my laptop this example runs to completion in ~70 seconds.
> \`\`\`
> julia: symbol lookup error: /lustre/home/pkrysl/a64fx/depot/artifacts/58dcf187642cdfbafb3581993ca3d8de565acc78/lib/openmpi/mca\_pmix\_pmix3x.so: undefined symbol: opal\_libevent2022\_evthread\_use\_pthreads
> \--------------------------------------------------------------------------
> Primary job terminated normally, but 1 process returned
> a non-zero exit code. Per user-direction, the job has been aborted.
> \--------------------------------------------------------------------------
> \[1972752\] signal (15): Terminated
> in expression starting at /lustre/home/pkrysl/a64fx/FinEtoolsDDParallel.jl/examples/heat/Poisson2D\_cg\_mpi\_driver.jl:160
> \_ZN12\_GLOBAL\_\_N\_117InterleavedAccess13runOnFunctionERN4llvm8FunctionE at /lustre/software/julia/julia-1.10.3/lib/julia/libLLVM-15jl.so (unknown line)
> \_ZN12\_GLOBAL\_\_N\_117InterleavedAccess13runOnFunctionERN4llvm8FunctionE at /lustre/software/julia/julia-1.10.3/lib/julia/libLLVM-15jl.so (unknown line)
> unknown function (ip: (nil))
> Allocations: 14621931 (Pool: 14605380; Big: 16551); GC: 19
> julia: symbol lookup error: /lustre/home/pkrysl/a64fx/depot/artifacts/58dcf187642cdfbafb3581993ca3d8de565acc78/lib/openmpi/mca\_pmix\_pmix3x.so: undefined symbol: opal\_libevent2022\_evthread\_use\_pthreads
> julia: symbol lookup error: /lustre/home/pkrysl/a64fx/depot/artifacts/58dcf187642cdfbafb3581993ca3d8de565acc78/lib/openmpi/mca\_pmix\_pmix3x.so: undefined symbol: opal\_libevent2022\_evthread\_use\_pthreads
> \--------------------------------------------------------------------------
> mpiexec detected that one or more processes exited with non-zero status, thus causing
> the job to be terminated. The first process to do so was:
> Process name: \[\[7833,1\],1\]
> Exit code: 127
> \`\`\`

---

<div class="post-metadata">

**Author:** ![farolap](https://avatars.discourse-cdn.com/v4/letter/f/ea666f/32.png) [@farolap](https://discourse.julialang.org/u/farolap)\
**Post date:** [September 1, 2025, 6:17am UTC](https://discourse.julialang.org/t/mpi-library-mismatch-erro-hdf5-netcdf-with-oceananigans/131579/3 "2025-09-01T06:17:17Z")

</div>

Thanks, Giordano! That solution solved my issue!