Got good results for some tasks and got an error for other tasks in a job array?

Specific Domains Julia at Scale
1

I am using my university HPC cluster and I have a job array to do some similar optimization problems using the package Optim.jl. Tasks should use the same packages and etc. They should only differ in the starting values. For some of the tasks in the job array, I get good output with no error. For other tasks, I got the following error. I really appreciate if someone can tell me what it means.

ERROR: LoadError: SystemError: opening file "/home/xz49/.julia/compiled/v1.4/Arpack_jll/m4kdm_NhCHm.ji": Permission denied
Stacktrace:
 [1] systemerror(::String, ::Int32; extrainfo::Nothing) at ./error.jl:168
 [2] #systemerror#50 at ./error.jl:167 [inlined]
 [3] systemerror at ./error.jl:167 [inlined]
 [4] open(::String; read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Bool) at ./iostream.jl:254
 [5] open(::String, ::String) at ./iostream.jl:310
 [6] open(::Base.var"#692#694", ::String, ::Vararg{String,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:296
 [7] open at ./io.jl:296 [inlined]
 [8] compilecache(::Base.PkgId, ::String) at ./loading.jl:1264
 [9] _require(::Base.PkgId) at ./loading.jl:1029
 [10] require(::Base.PkgId) at ./loading.jl:927
 [11] require(::Module, ::Symbol) at ./loading.jl:922
 [12] include(::Module, ::String) at ./Base.jl:377
 [13] top-level scope at none:2
 [14] eval at ./boot.jl:331 [inlined]
 [15] eval(::Expr) at ./client.jl:449
 [16] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Arpack/o35I5/src/Arpack.jl:8
ERROR: LoadError: LoadError: Failed to precompile Arpack [7d9fca2a-8960-54d3-9f78-7d1dccf2cb97] to /home/xz49/.julia/compiled/v1.4/Arpack/X5VZL_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] include(::String) at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:1
 [8] top-level scope at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:52
 [9] include(::Module, ::String) at ./Base.jl:377
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:331 [inlined]
 [12] eval(::Expr) at ./client.jl:449
 [13] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/pdsparsemat.jl:1
in expression starting at /home/xz49/.julia/packages/PDMats/HCniD/src/PDMats.jl:51
ERROR: LoadError: Failed to precompile PDMats [90014a1f-27ba-587c-ab20-58faa44d9150] to /home/xz49/.julia/compiled/v1.4/PDMats/wuzEE_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/xz49/.julia/packages/Distributions/dTXqn/src/Distributions.jl:3
ERROR: LoadError: Failed to precompile Distributions [31c24e10-a181-5473-b8eb-7969acd0382f] to /home/xz49/.julia/compiled/v1.4/Distributions/xILW0_NhCHm.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] exec_options(::Base.JLOptions) at ./client.jl:288
 [8] _start() at ./client.jl:484
in expression starting at /home/xz49/nots_consultant.jl:11
2

If feasible, I would check the permissions of

from a shell. It may be possible that these files were created when the process was running as a different user. These things depend on how your cluster is operated — you may want to ask local IT for support or read the relevant documentation; it is unlikely that it is a Julia issue per se.

Once you got things sorted out, you should make sure everything has the right permissions (chown -R / chmod -R) and try again. You can also try deleting compiled/cached files, see the discussion at

3

Hi, thank you for your suggestion. I checked the permission: -r–r–r. It might be because I updated some packages when the code was running in the cluster. I will try again.