Skip Package Precompile

General Usage
1

I’m working on a cluster computer that has two nodes:

  1. a login node with low resources (memory, cpu)
  2. a compute node with no internet acess

For this reason, it seems like the best workflow is to download packages on the login node and then precompile them on the compute node. When trying to precompile on the login node, it seems to crash due to low resources.

Is there a way to split Pkg.add("") into these stages? Best I can tell, it’s all baked in together. So for now I just run it twice and let it error out on the attempted login node precompile.

If there’s a better workflow please let me know.

1 Like
2

Try export JULIA_NUM_PRECOMPILE_TASKS=1 before running Julia on a login node. This prevents the precompile task from using more than one threads, which sometimes can lead to the process being killed due to the resource limit.