What I wanted to do was run multiple external commands simultaneously and collect the results…so my first thought was to use Distributed creating N workers and have them execute the commands and return the data via a RemoteChannel. But then I got to wondering about using a run(cmd) in a task, and if I could have multiple tasks running at the same time.
So I wrote a short test:
function test(i)
println("Starting $(i)")
run(`sleep 10`)
println("Finished $(i)")
end
for i in 1:10
@async test(i)
end
And to my surprise they all run at the same time, which I didn’t really expect.
So my question is which library method can cause a “task” switch? If I read from a file, could a task switch occur? What about reading from a network socket? I know yield() can cause a task switch, but have no clue what other operations may. And it doesn’t look like the documentation for run() mentions that it yields or could yield.
At the core of this program I’m working on I have a set of methods that abstract random read/write access to a file, and it would really screw me up if those methods “yielded” in the middle of execution. Or should I just put them all in a big mutex even when I’m not using threads? Do mutexes even honor the Task scheduling!?!?!?
I suspect this might just be teething issue, i.e. once threads are out of the experimental stage it will be YES you need a mutex you idiot. However at the moment the julia process is single threaded which usually means you don’t need to “worry” about the atomic operations and locking…unless like I said a method yields when you don’t expect it to…