[ANN] Malt.jl – Distributed alternative (used internally by Pluto)

We are happy to announce Malt.jl 1.0 – simple multiprocessing for Julia. Malt allows the creation and remote control of other Julia “worker” processes. Pluto uses Malt behind the scenes, using a Malt worker for every notebook to execute code.

Basic example

julia> import Malt

julia> worker = Malt.Worker();

julia> Malt.remote_eval_fetch(worker, :(1 + 1))

julia> Malt.remote_eval_fetch(worker, :(rand(5))) |> sum

Malt vs Distributed

Malt is inspired by the Distributed standard library, but with a focus on process sandboxing, not distributed computing. Important differences are:

API changes

Malt has different function names, see our documentation.

One important addition is public API for evaluating an Expr:

worker = Malt.Worker()

Malt.remote_eval_fetch(worker, :(sqrt(123)))

Nested use

With Malt, any worker process can also be a host process to its own workers.

In Distributed, only “process 1 can add or remove workers”. Malt does not have this limiation. This means that Malt workers can use Distributed (and Malt) like a regular Julia process.

Process isolation

Malt worker processes do not inherit ENV variables, command-line arguments or the Pkg environment from their host.

Interrupt on Windows

Malt supports interrupting a worker process on Windows, not just on UNIX.

No heterogenous computing

Malt does not have API like @everywhere or Distributed.procs: Malt is not the right tool for heterogenous computing.

Exception handling

Exceptions in Malt workers are converted to plaintext before being rethrown in the host.

The original exception object is only available to the worker. In Distributed, the original exception object is serialized and rethrown to the host.

Faster launch

Malt launches workers >50% faster.

julia> @time Distributed.addprocs(1);

2.064801 seconds (11.63 k allocations: 1.093 MiB, 1.08% compilation time)

julia> @time Malt.Worker();

0.964955 seconds (537 allocations: 308.734 KiB)

Thank you!

Malt was created by the awesome Sergio A. Vargas (@savq) during GSoC 2022, under mentorship of Paul Berg (@Pangoraw), inspired by the original Distributed stdlib. Nehal Patel (@habemus-papadum) was very helpful with the tricky networking and performance work, and JuliaHub sponsored me to complete the work and integrate the package with Pluto.jl.


Very glad to see Windows process interrupt - thanks to the Pluto team for addressing a real pain point for those of us stuck on Windows! This should significantly reduce the need to kill and restart Pluto notebooks.


Looks cool - when did pluto start using this?

1 Like

Pluto did a gradual roll-out of Malt:

  • 18 Sep 2023 – available as option with Pluto.run(workspace_use_distributed_stdlib=false)
  • 31 Oct 2023 – enabled by default on Windows
  • 15 Jan 2024 – enabled by default on all systems