We are happy to announce Malt.jl 1.0 – simple multiprocessing for Julia. Malt allows the creation and remote control of other Julia “worker” processes. Pluto uses Malt behind the scenes, using a Malt worker for every notebook to execute code.
Basic example
julia> import Malt
julia> worker = Malt.Worker();
julia> Malt.remote_eval_fetch(worker, :(1 + 1))
2
julia> Malt.remote_eval_fetch(worker, :(rand(5))) |> sum
3.0618168580350966
Malt vs Distributed
Malt is inspired by the Distributed standard library, but with a focus on process sandboxing, not distributed computing. Important differences are:
API changes
Malt has different function names, see our documentation.
One important addition is public API for evaluating an Expr:
worker = Malt.Worker()
Malt.remote_eval_fetch(worker, :(sqrt(123)))
Nested use
With Malt, any worker process can also be a host process to its own workers.
In Distributed, only “process 1 can add or remove workers”. Malt does not have this limiation. This means that Malt workers can use Distributed (and Malt) like a regular Julia process.
Process isolation
Malt worker processes do not inherit ENV variables, command-line arguments or the Pkg environment from their host.
Interrupt on Windows
Malt supports interrupting a worker process on Windows, not just on UNIX.
No heterogenous computing
Malt does not have API like @everywhere or Distributed.procs: Malt is not the right tool for heterogenous computing.
Exception handling
Exceptions in Malt workers are converted to plaintext before being rethrown in the host.
The original exception object is only available to the worker. In Distributed, the original exception object is serialized and rethrown to the host.
Faster launch
Malt launches workers >50% faster.
julia> @time Distributed.addprocs(1);
2.064801 seconds (11.63 k allocations: 1.093 MiB, 1.08% compilation time)
julia> @time Malt.Worker();
0.964955 seconds (537 allocations: 308.734 KiB)
Thank you!
Malt was created by the awesome Sergio A. Vargas (@savq) during GSoC 2022, under mentorship of Paul Berg (@Pangoraw), inspired by the original Distributed stdlib. Nehal Patel (@habemus-papadum) was very helpful with the tricky networking and performance work, and JuliaHub sponsored me to complete the work and integrate the package with Pluto.jl.