How to change code on workers - Revise.jl not able

Specific Domains Julia at Scale
1

I have a custom package that runs on multiple processes. If I addprocs(n); using Mypkg it will load the code to each worker (I can tell because of the annoying Nullable warnings).

Now when I edit the source code, my Revise.jl will change the code on my REPL, worker 1, but not on the parallel workers. The only way I have found to fix this is to remove the workers, add them and re-use:

map(rmprocs, workers())
addprocs(n)
using Mypkg

This takes a long time, but beats restarting. It seems to me that an @everywhere clever-revise-command would do the trick. Anyone have a clever way to do this?

Separately, I noticed that Revise.jl doesn’t revise when you start with julia -p n I submitted an issue for that.

Thanks in advance.

2

I just read the Revise.jl code. The only thing exposed is revise() and just tried this to no avail. The below code runs without error, but does not update the packages loaded on the workers.

@everywhere using Revise
@everywhere revise()
3

try

revise(module_name)
4

That’s getting closer - when I run @everywhere revise(Mypkg) it generates lots of warnings about methods being overwritten (a sign that Revise is working). I think it’s possible that those messages are all from worker 1 in the REPL.

But, the updated worker code info("Thank you Tamas!") does not appear in worker output until I do the worker remove/add/refresh. So close!

5

@tim.holy is amazing! He patched Revise.jl so to enable the code updates on remote workers. It’s only on 0.7 right now, but it’s nice to know it’s fixed, and maybe I’ll patch 0.6 with bits of that code.

4 Likes