While I am not new to distributed computing, I have trouble wrapping my head around how to implement the following behavior in Julia.
I have a testing tool that takes a bunch of tests (= directories with testing information), runs the tests, and displays the results on the terminal. Since all tests are serial, I would like to parallelize this in the following way (roughly):
- one central tasks/thread/worker (not sure about the correct terminology, let’s call it “root”) distributes the tests among the available CPU cores
- on each core, a single test is run concurrently
- once a test has finished, the result should be displayed on the terminal (with actual output being controlled by root), and the now idle core picks up the next test
- continue until all tests are finished
What would be the Julian way of implementing something like this? Or is this even how you would achieve the desired behavior (tests running on all cores, one core controlling I/O) with Julia?
I tried reading the official docs, but there are so many ways highlighted for doing something in parallel, I am not sure which one to pursue. Also, most tutorials/documentations I found online refer to pre-1.3 Julia (or even pre-1.0), thus I am sure they are at least in parts outdated.
Any help would be highly appreciated, even if it’s just pointing me to a good reference implementation or tutorial!