Always encounter this error after installing new packages on HPC?

Each time after I add a new Julia package on HPC, if I do not remove all the compiled packages in the folder “.julia/compiled” and then recompile, I will run into some random bugs. This is really annoying and I wonder why this happens. For example, I added a new package StatsBase and forgot to remove and recompile. Then I submitted a job with a job array of 10 independent and nearly identical “sub-jobs”. Six of them run perfectly and four of them report the following error. I really appreciate if someone can let me know why this happens. Since the ten jobs are identical and independent, if there is an error, I believe they should all report this error. But it seems that only some of the jobs report this error.

ERROR: LoadError: LoadError: SystemError: opening file "/home/xz49/.julia/compiled/v1.4/OpenBLAS_jll/vLkFP_NhCHm.ji": Permission denied
Stacktrace:
 [1] systemerror(::String, ::Int32; extrainfo::Nothing) at ./error.jl:168
 [2] #systemerror#50 at ./error.jl:167 [inlined]
 [3] systemerror at ./error.jl:167 [inlined]
 [4] open(::String; read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Bool) at ./iostream.jl:254
 [5] open(::String, ::String) at ./iostream.jl:310
 [6] open(::Base.var"#692#694", ::String, ::Vararg{String,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:296
 [7] open at ./io.jl:296 [inlined]
 [8] compilecache(::Base.PkgId, ::String) at ./loading.jl:1264
 [9] _require(::Base.PkgId) at ./loading.jl:1029
 [10] require(::Base.PkgId) at ./loading.jl:927
 [11] require(::Module, ::Symbol) at ./loading.jl:922
 [12] include(::Module, ::String) at ./Base.jl:377
 [13] include(::String) at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:1
 [14] top-level scope at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:50
 [15] include(::Module, ::String) at ./Base.jl:377
 [16] top-level scope at none:2
 [17] eval at ./boot.jl:331 [inlined]
 [18] eval(::Expr) at ./client.jl:449
 [19] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/wrappers/x86_64-linux-gnu-libgfortran4.jl:4
in expression starting at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:43
ERROR: LoadError: Failed to precompile Arpack_jll [68821587-b530-5797-8361-c406ea357684] to /home/xz49/.julia/compiled/v1.4/Arpack_jll/m4kdm_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Arpack/o35I5/src/Arpack.jl:8
ERROR: LoadError: LoadError: Failed to precompile Arpack [7d9fca2a-8960-54d3-9f78-7d1dccf2cb97] to /home/xz49/.julia/compiled/v1.4/Arpack/X5VZL_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] include(::String) at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:1
 [8] top-level scope at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:52
 [9] include(::Module, ::String) at ./Base.jl:377
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:331 [inlined]
 [12] eval(::Expr) at ./client.jl:449
 [13] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/pdsparsemat.jl:1
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:51
ERROR: LoadError: Failed to precompile PDMats [90014a1f-27ba-587c-ab20-58faa44d9150] to /home/xz49/.julia/compiled/v1.4/PDMats/wuzEE_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Distributions/fBLtF/src/Distributions.jl:3
ERROR: LoadError: Failed to precompile Distributions [31c24e10-a181-5473-b8eb-7969acd0382f] to /home/xz49/.julia/compiled/v1.4/Distributions/xILW0_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] exec_options(::Base.JLOptions) at ./client.jl:288
 [8] _start() at ./client.jl:484
in expression starting at /home/xz49/jpe.jl:16
srun: error: bc12u33n1: task 0: Exited with exit code 1
1 Like