Fwiw I haven’t found anything better since that post for using MPI transport in a notebook.
As far as using MPI to spawn workers, in a notebook I generally use ElasticManager
from ClusterManagers
and just srun
/mpiexec
the worker-connect-command with as many workers as I want, which has been very robust, as long as you can live with TCP transport.