I’d like to run system calls sequentially in a for loop, waiting for last call to complete before running the next one.
The run(command) call is nested in a function (run_system_call in the example under) which is part of a package that is built with parallel execution in mind (which is fine in 99% of cases) that I’d preferably not modify, so I’d like to avoid the obvious workaround of using /wait in the command to solve the problem on the system side. BTW I’m using Windows, but the problem seems to lie in Julia so I guess it would also be encountered under Linux.
I am using the following code (exemplified):
for (ic,num) in enumerate(cases)
while *completion criterion not satisfied*
*update completion criterion*
That does not work. run_system_call is called without error but does not run the system command in practice, obviously leading to deadlock in the while loop. I’ve tried rearranging the for/while loops, it does not help. It seems julia waits for for/while loops (and likely any soft scope) to complete before effectively sending out the system calls, that must be put in a buffer.
Anyone knowing if there is a way to “flush” the system call buffer? Or an idea of workaround?
I don’t think there is some buffer which isn’t flushed in time.
You example doesn’t have enough information to have an idea, but have you used the wait-parameter of run?
search: run trunc truncate round rounding RoundUp RoundDown RoundToZero RoundingMode RoundNearest RoundFromZero
run(command, args...; wait::Bool = true)
Run a command object, constructed with backticks (see the Running External Programs section in the manual). Throws
an error if anything goes wrong, including the process exiting with a non-zero status (when wait is true).
If wait is false, the process runs asynchronously. You can later wait for it and check its exit status by calling
success on the returned process object.
When wait is false, the process' I/O streams are directed to devnull. When wait is true, I/O streams are shared with
the parent process. Use pipeline to control I/O redirection.
Yeah that’s the point. I could solve the problem by tweaking the run_system_call function, but I’d rather not modify the package. And anyway, my post is also to raise attention on this weird behaviour of Julia. There must be some kind of buffering, it is waiting for the loop to complete before the system calls take action.
The completion criterion is based on checking whether output files from the system call have been updated. Anyway, the system call should open a new command window for each case so I can see immediately whether the it has been effectively executed or not.
The command is never run with a shell. Instead, Julia parses the command syntax directly, appropriately interpolating variables and splitting on words as the shell would, respecting shell quoting syntax. The command is run as julia 's immediate child process, using fork and exec calls.
Indeed, sorry for not being thourough here. I’m using run to run a batch script that in turns runs start <process> in a new window. If I remove the while loop, the code will effectively run the batch script for each case in the for loop, and I’ll be left with multiple instances of that script running in parallel, each in its own window.
Thanks a lot, here is some complementary info: run_system_call.jl should contain something like run(´cmd start script.bat´)
and script.bat should contain something like start cmd @cmd /k echo "hello world"
that should open a new window printing hello world on each. But if it behaves in the same way as on my machine, you won’t see any window popping up.
Yes I understand your point, in reality that’s quite complex code that is run, the overly complex batch stuff is to get the stdout properly on separate windows in different cases. run(script.bat) will put everything in the julia shell, which I don’t want. That won’t happen if script.bat is in turn running start cmd /k (I agree the @cmd was not necessary), but that is not always so, depending on cases[ic]. In short there are good reasons to have it like this