I want to parallelize some code on multiple nodes and am considering MPI.jl
One thing that feels a bit suboptimal, is that MPI seems quite incompatible with the interactive workflow. Ideally, I would like to keep my VS-code based workflow. But there seems to be no documented way to start a julia session with more than one worker (I assume this is likely not possible).
Since I haven’t found too much online, I want to ask you guys: What is your preferred way of developing MPI code.
Do you do it the “old-fashioned” way, i.e. write a script, and the exectute that with mpiexecjl -n ... , or is there anything more convenient to skip repeated compilation?
I have also briefly considered Distributed.jl / Dagger.jl but I am less sure it is the right choice here, as it seems to be more designed to parallelizing pure functions…
I have no idea about your actual workflow, I am just interested in which feature/performance characteristic exactly you are missing from Distributed/Dagger.
I am not at that stage yet where i am missing anything, but I felt a bit discouraged by this statement im the documentation:
Normally, Dagger tasks should be functional and “pure”: never mutating their inputs, always producing identical outputs for a given set of inputs, and never producing side effects which might affect future program behavior. However, for certain codes, this restriction ends up costing the user performance and engineering time to work around.
I suppose the solution to it is written below, but the text makes it look like it is more a workaround. But maybe i am interpreting it too pessimistically:D