# HDF5 build failure on HPC

**URL:** <https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859>\
**Category:** New to Julia\
**Tags:** question, package\
**Created:** [March 10, 2021, 8:53am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859 "2021-03-10T08:53:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [March 10, 2021, 8:53am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859/1 "2021-03-10T08:53:26Z")

</div>

Hello everyone,

could you please spend a minute to look at these error message ? I am trying to install `MLDatasets` on an HPC cluster but I got the following error while building and using `HDF5`. I know that a library is missing but I cannot fix it. I don’t even know how can I ask for help from the administrator. Any suggestions ?

Thank you very much …

program file to install packages

```julia
ENV["JULIA_PKG_SERVER"]="https://mirrors.tuna.tsinghua.edu.cn/julia"
using Pkg
Pkg.build("HDF5") # error here
using HDF5 # error here
using SpecialFunctions
using Zygote
using JLD2
using PyCall
using CUDA
using Flux
using BSON
using MLDatasets
using Lathe

```

output file

```julia
--------------- PROGRAM OUTPUT ----------------
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibSSH2_jll [29816b5a-b9ab-546f-933c-edad1886dfa8]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibCURL_jll [deac9b47-8bc7-5906-a0fe-35ac56dc84c0]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibSSH2_jll [29816b5a-b9ab-546f-933c-edad1886dfa8]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibSSH2_jll [29816b5a-b9ab-546f-933c-edad1886dfa8]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for HDF5_jll [0234f1f7-429e-5d53-9886-15a909be8d59]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibCURL_jll [deac9b47-8bc7-5906-a0fe-35ac56dc84c0]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for LibSSH2_jll [29816b5a-b9ab-546f-933c-edad1886dfa8]
│ exception = Required dependency MbedTLS_jll [c8ffd9c3-330d-5841-b78e-0817d7145fa1] failed to load from a cache file.
└ @ Base loading.jl:1042
ERROR: LoadError: LoadError: could not load library "libhdf5-00e8fae8.so.200.0.0"
libhdf5-00e8fae8.so.200.0.0: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Libdl/src/Libdl.jl:109
 [2] dlopen at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
 [3] top-level scope at /csns_workspace/CSG/lianyl/.julia/packages/HDF5/iH4LA/src/api_types.jl:119
 [4] include(::Function, ::Module, ::String) at ./Base.jl:380
 [5] include at ./Base.jl:368 [inlined]
 [6] include(::String) at /csns_workspace/CSG/lianyl/.julia/packages/HDF5/iH4LA/src/HDF5.jl:1
 [7] top-level scope at /csns_workspace/CSG/lianyl/.julia/packages/HDF5/iH4LA/src/HDF5.jl:51
 [8] include(::Function, ::Module, ::String) at ./Base.jl:380
 [9] include(::Module, ::String) at ./Base.jl:368
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:331 [inlined]
 [12] eval(::Expr) at ./client.jl:467
 [13] top-level scope at ./none:3
in expression starting at /csns_workspace/CSG/lianyl/.julia/packages/HDF5/iH4LA/src/api_types.jl:119
in expression starting at /csns_workspace/CSG/lianyl/.julia/packages/HDF5/iH4LA/src/HDF5.jl:51
   Building HDF5 → `~/.julia/packages/HDF5/iH4LA/deps/build.log`
ERROR: LoadError: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to /csns_workspace/CSG/lianyl/.julia/compiled/v1.5/HDF5/L7Dga_3RSiF.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] exec_options(::Base.JLOptions) at ./client.jl:296
 [9] _start() at ./client.jl:506
in expression starting at /csns_workspace/CSG/lianyl/test_CUDA.jl:17
--------------- PROGRAM COMPLETED--------------

```

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [March 10, 2021, 8:56am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859/2 "2021-03-10T08:56:59Z")

</div>

strangely enough, when I build `HDF5` (and then `MLDatasets`) from the login node, no error has ever occurred.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [March 10, 2021, 9:08am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859/3 "2021-03-10T09:08:01Z")

</div>

In most (not all) HPC cluster architectures the cluster compute nodes are on a private network.  
Normally they access the Internet by a NAT through the cluster head node (there are other architectures)  
Cluster head nodes may have to contact license servers also, via a NAT (Network Address Translation)  
It may be that on your cluster the cluster compute nodes cannot access servers outside the cluster.

What does the build log contain?  
~/.julia/packages/HDF5/iH4LA/deps/build.log

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [March 10, 2021, 9:08am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859/4 "2021-03-10T09:08:54Z")

</div>

Please do ask your cluster admins for advice.  
I recommend bringing cookies. Throw the cookies between the bars of their cage and they will be like pussy cats.

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [March 11, 2021, 12:41am UTC](https://discourse.julialang.org/t/hdf5-build-failure-on-hpc/56859/5 "2021-03-11T00:41:48Z")

</div>

Meow