How to properly run julia 1.9 on LSF cluster

General Usage
, ,
1

I updated to julia 1.9.3, and I am unable to run my code on a cluster.

The 1st issue is the precompilation. It seems to me that every time I open a new session with the cluster, I need to precompile my project. As far as I understand, this is a julia 1.9 thing. Is there a way to deal with this when working on a cluster?

The 2nd issue, is a segmentation fault.
Running my script I see these warnings:

<E2><94><8C> Warning: The call to compilecache failed to create a usable precompiled cache file for FileIO [5789e2e9-d7fb-5bc7-8068-2c6fae9b9549]
<E2><94><82>   exception = ArgumentError: Invalid checksum in cache file /home/labs/orenraz/roiho/.julia/compiled/v1.9/FileIO/6iKRU_gncnE.so.
<E2><94><94> @ Base loading.jl:1818

and

<E2><94><94> @ Base loading.jl:1793
<E2><94><8C> Warning: Module FileIO with build ID ffffffff-ffff-ffff-001c-a130a3b6bf6f is missing from the cache.
<E2><94><82> This may mean FileIO [5789e2e9-d7fb-5bc7-8068-2c6fae9b9549] does not support precompilation but is imported by a module that does.

Followed by

/scratch/1697361899.830137.65: line 8: 58683 Segmentation fault      (core dumped)

I posted this question before, but I phrased it badly, and that is why I opened this new thread.

I’d love to know if there is a “good” way to run julia 1.9 on a cluster or if the best thing is to go back to julia 1.7 (which is the version I had before and was working well).

2

Are you running with distributed workers or what does running on a cluster entail?

Can you try to run it locally with a few workers on you computer and see if you run into the same problem?

Can you try to delete the ~/.julia/compiled folder and instantiate everything again? Have had similar problems that seemingly came from some borked compilation, and were solved by just removing and instantiating.

1 Like
3

I am just running many instances of the same script, so I am not dealing with distributed workers or anything like that.

Using the tip from this thread setting

export JULIA_CPU_TARGET="generic;icelake-server,clone_all"

I got a bit ahead. I still get the warnings I mentioned above, but there is no segmentation fault. I am still waiting for things to finish to see how they turn out.

If things do not go well, I will try to delete the ~/.julia/compiled folder and instantiate again.

Will update soon! Thanks!

4

See this post for the solution. Thanks!