Installing Julia Packages on a cluster

General Usage
1

Hi,

I’ve just started using Julia on my cluster account. Julia has to be recently installed by the admin and there isn’t much support from them with regards to usage. I can submit and run simple jobs, however when I try to install packages, I get several errors:

Resolving package versions…
ERROR: LoadError: Error when installing package Rmath:
IOError: sendfile: Unknown system error -122 (Unknown system error -122)

Cluster admin says it probably has to do with a permissions problem, and “when installing your own packages you need to specify somewhere you can write to, instead of the system directory”

Does anyone have an idea how to go about this?

2

When using Julia on a cluster I’ve found it easier to install it myself rather than relying on a central installation.

You should be able to do this if you download Julia into an area you can write to, e.g. your home directory, and unpack it there. Then call that binary to access Julia.

Pkg.add() should then work provided the cluster can access the internet and the firewall settings are okay.

1 Like
3

Do you have write access to the DEPOT_PATH?

4

I will back up what @jgreener64 says.
With the catch though - many times on HPC clusters your home directory is on slower NFS storage, and you often have a very small quota. If you start a parallel job on thousands of nodes with a dynamically linked language which used shared libraries you end up reading those libraries many times at once - which puts a big strain on the NFS filesystem.
Thats why there are fast filesystems.

2 Likes
5

I was going to say - I normally install julia to /usr/local/julia-1.5 (say)
and then set a link /usr/local/julia → /usr/local/julia-1.5
And add /usr/local/julia/bin to my PATH
(yes I should use Modules)
I have never had a problem with permissions like you see.
Backing up @hendri54 can you give is the results of ENV at your prompt? (assuming you run Linux)

1 Like
6

Hey! I am trying to run Julia on an HPC, so before I start just wanted to know if I will be able to submit and run Julia codes if I have installed Julia by myself in my directory and has not been installed by the admin?
And if yes , then is there any particlular way to do so ?
P.S.~ I’m quite new to this things, so I do not have mush of expereince! :confused: