[ANN] DistSSHKit.jl: run a Julia project on multiple machines over SSH

Package Announcements
, , , ,
1

Hi everyone,

I’d like to announce DistSSHKit.jl, recently added to the General registry.

 pkg> add DistSSHKit

DistSSHKit runs the same Julia project locally and over SSH, then collects the results. Uses Distributed.jl processes, not threads. Supported on macOS, Linux, and WSL2 Ubuntu (not native Windows), Julia 1.12+.

It handles the whole loop: deploys your project to each remote (via --rsync or --git), installs dependencies, runs your script, and collects the results — no scheduler and no manual per-machine setup. Two ways to run: go treats each machine as an independent, complete job; drive has a single master farm work out to all of them.

DistSSHKit topology
DistSSHKit topology1120×472 34.7 KB

# Prepare a remote
julia --project=. -m DistSSHKit setup --rsync user@host1
julia --project=. -m DistSSHKit setup --instantiate user@host1

# go: run script.jl as-is on each host
julia --project=. -m DistSSHKit go user@host1:1 path/to/script.jl

# drive: one master farms work to local and remote workers
julia --project=. -m DistSSHKit drive local:2 user@host1:4 path/to/driver.jl

Assumes SSH access to each remote is already set up, and that Julia is installed on every machine (local and remote) beforehand — DistSSHKit deploys your project and dependencies, not Julia itself. Also needs ssh, rsync, and (git deploy only) git — pkg> add won’t install these.

Docs: Home · DistSSHKit.jl

The v0.3 CLI and API aren’t changing for now — we’re just fixing bugs. Separately, we’re also working on DistSSHQueue.jl, a lightweight scheduler that should make your SSH machines feel even more like real compute nodes.

Built this to make use of our lab’s Mini PCs that sit idle most of the time, and more broadly, to make Julia’s computing environment feel more integrated and approachable for small labs.

We’re also looking for help with maintenance — feedback and contributions welcome.

2

Update: 0.4 (2026-08-24) broke the examples above. The lightweight scheduler from the original post (DistSSHKitQueue.jl, still not in General) needed the same placement vocabulary as the kit, so we cut 0.4 instead of keeping the v0.3 freeze. Sorry for the churn.

go / drive tokens are now parent[:N] and child:NAME[:N]. local:2 / bare user@host are gone. setup still takes a bare SSH name.

julia --project=. -m DistSSHKit go child:user@host1:1 path/to/script.jl
julia --project=. -m DistSSHKit drive parent:2 child:user@host1:4 path/to/driver.jl

docs · NEWS

3

I plan to make a separate announcement, but since there have been significant changes, I will add them to this thread. Also, automatic translation has recently been added. I would also like to share some thoughts on the development.

Currently, DistSSHKit (v0.10) has incorporated DistSSHRun, which provides execution capabilities, and the newly added DistSSHQueue, which can queue jobs using resident machines for sequential processing. It has become a toolkit that more powerfully supports distributed processing across Julia machines.

Development is still very active. However, what I originally intended to do is being steadily achieved. As for myself, I conduct research on evolutionary games and human social behavior using agent-based simulations. In many social science laboratories, sufficient computing resources are often unavailable. During development, I consulted with researchers from various laboratories about their computing environments.

Even in such small laboratories, our goal is to connect mini-machines available locally and utilize them easily as computing resources. I envision a future where anyone could wield supercomputing power (although simply performing distributed processing via SSH does not technically qualify these systems as supercomputers). After all, there are increasingly many small yet powerful computers nowadays.

We will continue to develop energetically. In fact, we intend to verify the tools repeatedly alongside our own research. Thank you for your continued support!