# Precompilation error

**URL:** <https://discourse.julialang.org/t/precompilation-error/92607>\
**Category:** New to Julia\
**Tags:** package, precompilation\
**Created:** [January 6, 2023, 3:44pm UTC](https://discourse.julialang.org/t/precompilation-error/92607 "2023-01-06T15:44:17Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 6, 2023, 3:44pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/1 "2023-01-06T15:44:17Z")

</div>

Hi!  
I am using a package written in Julia language, which is very new to me. I am using Julia v.1.7.2 on HPC cluster. I notice a very strange error which appears randomly when I submit multiple jobs with different parameters.

> ERROR: LoadError: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to /home/a/as42/.julia/compiled/v1.7/HDF5/jl\_d6InQ8.

I get this error for multiple packages even though in the log file I see :

> .julia/compiled/v1.7/HDF5/

I understand this is caused by multiple codes trying to access the base compiled file. I am not sure how to fix this issue.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [January 6, 2023, 7:23pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/2 "2023-01-06T19:23:48Z")

</div>

If you run the following do you get a more extensive error message?

```julia
using Pkg
Pkg.precompile()

```

My suspicion is that LD\_PRELOAD or LD\_LIBRARY\_PATH environment variables may be set causing Julia to load a libhdf5.so that it is not compatible with.

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 6, 2023, 10:48pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/3 "2023-01-06T22:48:23Z")

</div>

Hi!  
Thank you for your reply. I checked that I did not get any error message when I run the commands suggested by you.  
I have now reinstalled everything again to see if it helps.  
However, I am still getting the following warning messages:

> Warning: Module HDF5 with build ID 18931272587474617 is missing from the cach e.  
> │ This may mean HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] does not support pr ecompilation but is imported by a module that does.  
> └ @ Base loading.jl:1325  
> ┌ Warning: The call to compilecache failed to create a usable precompiled cache file for ITensors [9136182c-28ba-11e9-034c-db9fb085ebd5]  
> │ exception = Required dependency Zlib\_jll [83775a58-1f1d-513f-b197-d71354ab0 07a] failed to load from a cache file.  
> └ @ Base loading.jl:1349  
> I believe this sometimes leads to killing the job. I don’t understand how to fix this.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [January 6, 2023, 11:17pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/4 "2023-01-06T23:17:29Z")

</div>

I think something got borked in your precompile cache. Delete the `/home/a/as42/.julia/compiled/v1.7` directory and try again.

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [January 7, 2023, 8:12pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/5 "2023-01-07T20:12:39Z")

</div>

If the other suggestions don’t work, you may need to load in the environment which gives support for `HDF5` in linux. I suggest looking for it like:

```plaintext
module spider hdf5

```

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 12, 2023, 10:20pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/6 "2023-01-12T22:20:56Z")

</div>

Can you elaborate on how to implement this?  
I am still getting these errors on Julia 1.8.4.  
I am submitting multiple jobs at once on HPC and I randomly get some this precompile error specific to HDF5 package.

```julia
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for HDF5_jll [0234f1f7-429e-5d53-9886- 15a909be8d59]
 67 │ exception = Required dependency Zlib_jll [83775a58-1f1d-513f-b197-d71354ab007a] failed to load from a cache file.
└ @ Base loading.jl:1349
ERROR: LoadError: could not load symbol "H5open":
/home/as42/julia-1.8.4/bin/julia: undefined symbol: H5open
 

ERROR: LoadError: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to /home/a/as42/.julia/compiled/v1.8/HDF 5/jl_NIxl0D.

```

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [January 12, 2023, 10:40pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/7 "2023-01-12T22:40:08Z")

</div>

I don’t know whether my suggestion will work, but I assume that your cluster admins will have probably compiled HDF5 and made it available as a module that you can optionally load in. If you type

```plaintext
module avail

```

into your shell, you should see some options. `module spider` allows you to search for a package. This command just checks whether your admins have made HDF5 available. On both clusters I have access to, there are multiple versions of HDF5 installed. Try to load the module with

```plaintext
module load HDF5/1.13.1

```

If this doesn’t work, type `module spider HDF5/1.13.1` (replace the name with the one/version on your system). This will tell you all the modules needed to load. On my system, I need 4 other packages:

```plaintext
module load GCC/11.3.0
module load OpenMPI/4.1.4 
module load intel-compilers/2022.1.0 
module load impi/2021.6.0
module load HDF5/1.13.1

```

Once you have these, put it in your slurm batch script before you run Julia, and all the workers should have access to the right modules. I _think_ this should help.

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 12, 2023, 10:53pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/8 "2023-01-12T22:53:39Z")

</div>

Thank you for the nice explanation. I am not much aware of the module functionality.  
When I type “module avail” on cluster, I get the following:

```julia
--------------------------------------------- /usr/share/modules/versions -------------------------------------------------
3.2.10

-------------------------------------------- /usr/share/modules/modulefiles ------------------------------------------------
dot module-git module-info modules null use.own
e

```

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [January 13, 2023, 8:32am UTC](https://discourse.julialang.org/t/precompilation-error/92607/9 "2023-01-13T08:32:02Z")

</div>

Have you tried `module spider hdf5`?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 13, 2023, 9:07am UTC](https://discourse.julialang.org/t/precompilation-error/92607/10 "2023-01-13T09:07:49Z")

</div>

Julia should use the BinaryBuilder-compiled version of libhd5 via [HDF5\_jll](https://juliapackages.com/p/hdf5_jll). I’d be surprised if anything would improve by trying to load some other hdf5 library.

@AS_92, to me this feels like you might have a precompilation race condition: some file is getting rewritten and that invalidates the cache for other packages. Just to clarify, if you execute the suggestion in [Precompilation error - #2 by mkitti](https://discourse.julialang.org/t/precompilation-error/92607/2) (_sequentially_ in each environment you depend on) and then don’t touch your package environment(s), does that fix the problem?

If that doesn’t fix the problem, then the follow-up is: what packages are you using? If they’re doing some nasty pkg work that could explain it (and that would be a bug in the package(s) you’re using rather than Julia itself). One way to diagnose that kind of bad behavior might be to do the `precompile` step as described above and then temporarily disable write permission recursively (all subdirectories and files) on your `~/.julia/compiled/v1.8` directory. Then you should get an immediate error if some package is going rogue.

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 13, 2023, 3:08pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/11 "2023-01-13T15:08:31Z")

</div>

Thank you for your reply. I am very new to Julia so I am not sure how to implement some of the things you have mentioned. It will be kind of you if you could help me with this.

I am not building a package in Julia, rather I am just using a package written in it. So, if I understand this correctly my environment is julia 1.8. When submit my codes on HPC, I am not initiating precompile. I am just adding the packages “using PKGNAME” in my code.jl.  
I set the path for julia in my bash file do the packages already compiled in /.julia/compiled/1.8 are linked to the code.

I agree I am running into race condition, as the error appears when the code is running on the same core (I am using multithreading, its possible my nodes are competing) and hence it is appearing randomly.

Should I precompile the packages every time I run a code?  
How do I disable the permission to write in the directory?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 13, 2023, 3:20pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/12 "2023-01-13T15:20:51Z")

</div>

Normally everything should just work. However, there seems to be something funny going on in your particular combination of packages and environment.

The first thing to try is to enter package mode (type `]` at the `julia>` prompt) and type `precompile`. After that finishes, hit backspace to go back to `julia>` mode and try running your code.

If that doesn’t fix it, it suggests something strange is happening. I’m proposing that you debug it by turning off write permissions on your compiled-package cache: something like `chmod -R -w ~/.julia/compiled/v1.8`, assuming you’re running unix. (Untested so you may have to experiment.) Now if you run your code, you might get a different error. Paste that error into this thread and perhaps it will shed light on what’s going wrong.

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 13, 2023, 10:50pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/13 "2023-01-13T22:50:58Z")

</div>

I tried to run the code using the bash file after removing the permissions:  
I got the following error:

```julia
 26 ERROR: LoadError: SystemError: mktemp: Permission denied
 27 Stacktrace:
 28 [1] systemerror(p::Symbol, errno::Int32; extrainfo::Nothing)
 29 @ Base ./error.jl:176
 30 [2] #systemerror#80
 31 @ ./error.jl:175 [inlined]
 32 [3] systemerror
 33 @ ./error.jl:175 [inlined]
 34 [4] mktemp(parent::String; cleanup::Bool)
 35 @ Base.Filesystem ./file.jl:623
 36 [5] mktemp
 37 @ ./file.jl:620 [inlined]
 38 [6] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
 39 @ Base ./loading.jl:1669
 40 [7] compilecache
 41 @ ./loading.jl:1651 [inlined]
 42 [8] _require(pkg::Base.PkgId)
 43 @ Base ./loading.jl:1337
 44 [9] _require_prelocked(uuidkey::Base.PkgId)
 45 @ Base ./loading.jl:1200
 46 [10] macro expansion
 47 @ ./loading.jl:1180 [inlined]
 48 [11] macro expansion
 49 @ ./lock.jl:223 [inlined]
 50 [12] require(into::Module, mod::Symbol)
 51 @ Base ./loading.jl:1144

```

Following some other suggestions on the discourse, I also tried doing the following thing:

```julia
julia --project -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'
julia -t $OMP_NUM_THREADS $ARG > out.dat

```

but I got the error:

```julia
29 ERROR: LoadError: SystemError: mktemp: Permission denied
 30 Stacktrace:
 31 [1] systemerror(p::Symbol, errno::Int32; extrainfo::Nothing)
 32 @ Base ./error.jl:176
 33 [2] #systemerror#80
 34 @ ./error.jl:175 [inlined]
 35 [3] systemerror
 36 @ ./error.jl:175 [inlined]
 37 [4] mktemp(parent::String; cleanup::Bool)
 38 @ Base.Filesystem ./file.jl:623
 39 [5] mktemp
 40 @ ./file.jl:620 [inlined]
 41 [6] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
 42 @ Base ./loading.jl:1669
 43 [7] compilecache
 44 @ ./loading.jl:1651 [inlined]
 45 [8] _require(pkg::Base.PkgId)
 46 @ Base ./loading.jl:1337
 47 [9] _require_prelocked(uuidkey::Base.PkgId)
 48 @ Base ./loading.jl:1200
 49 [10] macro expansion
 50 @ ./loading.jl:1180 [inlined]
 51 [11] macro expansion
 52 @ ./lock.jl:223 [inlined]
 53 [12] require(into::Module, mod::Symbol)
 54 @ Base ./loading.jl:1144

```

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 14, 2023, 10:17am UTC](https://discourse.julialang.org/t/precompilation-error/92607/14 "2023-01-14T10:17:58Z")

</div>

I tried to run the codes again after precompiling and giving the write permissions to compiled folder. I started a bunch of jobs using a bash script, I notice some of them are getting killed.  
The screenshot below shows the difference in the log files of two codes which were started at the same time. The job in the left screen (theta\_500) successfully finished while the one on the right was killed immediately. I notice both of them started at the same time on the same node. Additionally I notice, one major difference in the compiled folder ".julia/compiled/v1.8/HDF5\_jll/ ", this seems to be the main cause of the error.  
I also notice, the left job (theta\_500) throws a warning error, but the job is still successfully finished.

 ![Screenshot from 2023-01-14 10-59-53](https://global.discourse-cdn.com/julialang/original/3X/8/2/82dfe1024bdcd3130c07880036883e30e6190b33.png)

I notice the compiled folder imported by different jobs are different. when HDF5\_jll is missing, the job gets terminated immediately, but when it loads more files like in theta\_100 and theta\_200, I get errors in the later part of the code arising from a package I am using.

 ![Screenshot from 2023-01-14 11-13-38](https://global.discourse-cdn.com/julialang/original/3X/9/6/96feacf3e0a409fe78862c9aa36333f910d58a26.png)

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 18, 2023, 3:04pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/15 "2023-01-18T15:04:08Z")

</div>

I am not sure what is going on. One possible way to start diagnosing is to start by adding just the packages that seem to fail frequently to your environment, and precompile them:

```julia
pkg> precompile HDF5

```

If those fail, what’s the error? If it fails, you can get a more expansive error with

```julia
julia> using HDF5

```

You may need to do this for each “proximal” cause of failure. Then perhaps you can precompile your larger package?

---

<div class="post-metadata">

**Author:** ![AS\_92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/as_92/32/31405_2.png) [@AS\_92](https://discourse.julialang.org/u/AS_92)\
**Post date:** [January 18, 2023, 3:21pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/16 "2023-01-18T15:21:15Z")

</div>

precompiling HDF5 doesn’t seem to throw any error.  
I believe it throws the error only when two different codes on the same node tries to access the cache of HDF5 at the same time.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 18, 2023, 3:44pm UTC](https://discourse.julialang.org/t/precompilation-error/92607/17 "2023-01-18T15:44:35Z")

</div>

_Access_ shouldn’t cause that problem (emphasis on shouldn’t). Two nodes trying to _write_ the same cache file at the same time might. That’s why I wonder if precompiling serially might fix things.

One (slow) way to precompile everything might be to clear your `.julia/compiled/v1.x` directory and then just start with `julia> using MyBigPkg` _without_ first using `pkg> precompile`. That should force all the dependencies of `MyBigPkg` to be precompiled one-at-a-time. You’ll have to be patient, as this can take a long time, but then perhaps your parallel job can work?

This might not work, though, if something in `MyBigPkg` is doing `Pkg`-level operations. It probably shouldn’t be doing that, so you might complain to the author if that seems to be what’s happening.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [January 20, 2023, 7:09am UTC](https://discourse.julialang.org/t/precompilation-error/92607/18 "2023-01-20T07:09:25Z")

</div>

Why is `mktemp` failing though? Is more than one process trying to create the same temporary file?

Could you just run a single process first before trying to launch multiple?

Another option might be trying to give each process its own `JULIA_DEPOT_PATH` as a last resort.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 20, 2023, 11:16am UTC](https://discourse.julialang.org/t/precompilation-error/92607/19 "2023-01-20T11:16:28Z")

</div>

> Why is `mktemp` failing though?

Because I had specifically asked him to turn off write permissions to see _when_ a particular write step was occurring. There were additional writes after all the files were nominally precompiled, which is weird. [`compilecache` failed when `@everywhere using` from remote machines · Issue #48217 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/48217) seems similar. There’s a bug somewhere but we’re still hunting for it.