A short question. Is there some way to use pmap such that the following doesn’t throw an exception and crash? I have a distributed system, where a process might use too much memory and be killed by the os. This seems to result in a ProcessExitedException and the entire program crashing. I am, however, okay with individual batches failing and would like the program to continue.
using Distributed addprocs(3) @everywhere function calc(i) val = mod(first(rand(Int, 1)), 4) + 1 sleep(val) if myid() == 2 # Emulating Process Killed due to too much memory usage exit(1) end return i + val end result = pmap(calc, vcat(1, workers()); distributed=true, batch_size=1, retry_delays=, retry_check=nothing, on_error=ex->-1 ) print("Didn't crash")