Using Distributed with a mix of architectures

Tooling
1

I have a mix of Intel and Apple Silicon Macs that I would like combine into a loose “cluster”. It seems that using Distributed from the standard library is the way to go, but after setting up passwordless ssh login and making sure that all the machines have the same julia version and other packages installed, I get the following error when trying to start julia supplying a machine file:

julia --project=. --machine-file=machinefile.txt
sh: line 1: /Users/roger/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/bin/julia: No such file or directory
sh: line 1: exec: /Users/roger/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/bin/julia: cannot execute: No such file or directory

This presumably because on the Intel machines, the full julia path is /Users/roger/.julia/juliaup/julia-1.11.4+0.x64.apple.darwin14/bin/julia

Is there an established solution for this cross-architecture use of Distributed?

2

I manned to fix this by calling addprocs explicitly, supplying the executable name as a parameter.

using Distributed
params = (exename="/home/user/.juliaup/bin/julia", dir=pwd())
addprocs([("work1",1)];params...)

where work1 is a host with a corresponding entry in my ~/.ssh/config file.

2 Likes