I flailed around on a an approach resembling yours in my MWE here and hit the wall where there is no isopen()
function nor an open()
for RemoteChannel
. In that MWE code above, I had to detect that the channel was closed by catching the error and exiting the loop. Crude, and there was no way to restart the thing because there is no open()
Later, I set up code to put zero-flagged jobs onto the job queue, and the workers would exit when they saw a zero job. There was a bit of flushing the pipes but it worked.
Then, I abandoned the whole approach and went to a variant of the scheduler in the example code. This works better in my case, because emit()
was causing too much data traffic, and it was better to accumulate map results in the worker before returning a result. I’m happy with this method.
So, the answer is that you have to catch the error, but there may be more elegant ways to do it.