MPI on Raspberry pi distributed computer

Specific Domains Julia at Scale
,
1

Hello everyone. I’m a total beginner in Julia and cluster computing. I have already made a Raspberry pi distributed computer with 4 nodes. Any node can SSH to every other. Every node should perform some calculations “locally”. After completing calculations, these nodes should be able to communicate with an arbitrary number of nodes. And then this process of calculating and communicating should be repeated over again. I guess this is non-classic cluster computing. I have tested some functions in “Distributed” library and it worked fine, but I want to use MPI wrapper for Julia to perform message parsing.

I saw that in Python you can write something like this: “mpiexec -f machinefile -n 4 python script.py”. And it runs “script.py” on every node (nodes specified in “machinefile” with ip addresses). But, I couldn’t find how to do this in Julia. Closest thing I can find is “mpiexec -np 4 julia script.jl”, but this runs “script.jl” on 4 processes locally on the same node where the program is executed. Is it possible to run “script.jl” on all 4 nodes in Julia?

If not, what are you suggesting I do? What is the best way to do what I want? Every suggestion is welcome.

1 Like
2

Seems like part of the confusion is that this command isn’t “in python”, it’s a shell command that runs a python script. The only role python has is interpreting your script script.py. The rest of the command is calling mpiexec with the options -f machinefile and -n 4; that part doesn’t care what command you’re trying to run, just how many processes and which hosts you’d like to run it on.

In particular, the option -f machinefile, tells MPI what hosts (i.e. computers) you’d like the command to be run on (see here for more information in the case of openMPI; the other implementations are similar in behavior).

Examining the command you used to run your Julia script:

mpiexec -np 4 Julia script.jl

Note that the -f machinefile part of the command is missing. The script is only running on the local node because you haven’t told it to do otherwise.

1 Like