Each time after I add a new Julia package on HPC, if I do not remove all the compiled packages in the folder “.julia/compiled” and then recompile, I will run into some random bugs. This is really annoying and I wonder why this happens. For example, I added a new package StatsBase and forgot to remove and recompile. Then I submitted a job with a job array of 10 independent and nearly identical “sub-jobs”. Six of them run perfectly and four of them report the following error. I really appreciate if someone can let me know why this happens. Since the ten jobs are identical and independent, if there is an error, I believe they should all report this error. But it seems that only some of the jobs report this error.
ERROR: LoadError: LoadError: SystemError: opening file "/home/xz49/.julia/compiled/v1.4/OpenBLAS_jll/vLkFP_NhCHm.ji": Permission denied
Stacktrace:
[1] systemerror(::String, ::Int32; extrainfo::Nothing) at ./error.jl:168
[2] #systemerror#50 at ./error.jl:167 [inlined]
[3] systemerror at ./error.jl:167 [inlined]
[4] open(::String; read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Bool) at ./iostream.jl:254
[5] open(::String, ::String) at ./iostream.jl:310
[6] open(::Base.var"#692#694", ::String, ::Vararg{String,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:296
[7] open at ./io.jl:296 [inlined]
[8] compilecache(::Base.PkgId, ::String) at ./loading.jl:1264
[9] _require(::Base.PkgId) at ./loading.jl:1029
[10] require(::Base.PkgId) at ./loading.jl:927
[11] require(::Module, ::Symbol) at ./loading.jl:922
[12] include(::Module, ::String) at ./Base.jl:377
[13] include(::String) at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:1
[14] top-level scope at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:50
[15] include(::Module, ::String) at ./Base.jl:377
[16] top-level scope at none:2
[17] eval at ./boot.jl:331 [inlined]
[18] eval(::Expr) at ./client.jl:449
[19] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/wrappers/x86_64-linux-gnu-libgfortran4.jl:4
in expression starting at /home/xz49/.julia/packages/Arpack_jll/V7Qow/src/Arpack_jll.jl:43
ERROR: LoadError: Failed to precompile Arpack_jll [68821587-b530-5797-8361-c406ea357684] to /home/xz49/.julia/compiled/v1.4/Arpack_jll/m4kdm_NhCHm.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval at ./boot.jl:331 [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Arpack/o35I5/src/Arpack.jl:8
ERROR: LoadError: LoadError: Failed to precompile Arpack [7d9fca2a-8960-54d3-9f78-7d1dccf2cb97] to /home/xz49/.julia/compiled/v1.4/Arpack/X5VZL_NhCHm.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] include(::String) at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:1
[8] top-level scope at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:52
[9] include(::Module, ::String) at ./Base.jl:377
[10] top-level scope at none:2
[11] eval at ./boot.jl:331 [inlined]
[12] eval(::Expr) at ./client.jl:449
[13] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/pdsparsemat.jl:1
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:51
ERROR: LoadError: Failed to precompile PDMats [90014a1f-27ba-587c-ab20-58faa44d9150] to /home/xz49/.julia/compiled/v1.4/PDMats/wuzEE_NhCHm.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval at ./boot.jl:331 [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Distributions/fBLtF/src/Distributions.jl:3
ERROR: LoadError: Failed to precompile Distributions [31c24e10-a181-5473-b8eb-7969acd0382f] to /home/xz49/.julia/compiled/v1.4/Distributions/xILW0_NhCHm.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] exec_options(::Base.JLOptions) at ./client.jl:288
[8] _start() at ./client.jl:484
in expression starting at /home/xz49/jpe.jl:16
srun: error: bc12u33n1: task 0: Exited with exit code 1