I’m looking for a sort of dynamically sized cluster manager, but without the requirement for a dedicated head node that’s responsible for adding and removing worker nodes. I know of ElasticManager from ClusterManagers.jl, but that doesn’t quite fit the bill - the head node is still assumed to manage the dynamically coming and going workers.
I guess I’m looking for workers that self manage when other workers connect to them? In my specific use case, I can assume that I know the IPs of possible workers, so starting a worker and then trying to connect to “the mesh” by pinging each expected node about its own existence is a possibility.
Does anyone know if this sort of thing has been done before or if implementing this based on ElasticManager would be best?