Is there any way to find out the current status of each worker process, whether they are currently available or busy?
Each worker process might be running async tasks, so you cannot figure it out even if you monitor the communication channel between master and worker. One way might be to monitor CPU usage of the workers out of band.
To reliably do this, you’ll need some kind of task counting system. pmap does this somewhat (send a job when the previous job returns), and dagger also presumably does something similar.
Regards
Avik