Buffer in running external program in loops

For the discussion to be in sync, here again my guess:

The completion criterion is based on checking whether output files from the system call have been updated. Anyway, the system call should open a new command window for each case so I can see immediately whether the it has been effectively executed or not.

Perhaps not. run doesn’t open a CMD shell, it directly starts the commands using exec calls.

https://docs.julialang.org/en/v1.7/manual/running-external-programs/

  • The command is never run with a shell. Instead, Julia parses the command syntax directly, appropriately interpolating variables and splitting on words as the shell would, respecting shell quoting syntax. The command is run as julia 's immediate child process, using fork and exec calls.

Indeed, sorry for not being thourough here. I’m using run to run a batch script that in turns runs start <process> in a new window. If I remove the while loop, the code will effectively run the batch script for each case in the for loop, and I’ll be left with multiple instances of that script running in parallel, each in its own window.

Ok, I try to mimic this here on my windows machine…

Thanks a lot, here is some complementary info:
run_system_call.jl should contain something like
run(´cmd start script.bat´)
and script.bat should contain something like
start cmd @cmd /k echo "hello world"

that should open a new window printing hello world on each. But if it behaves in the same way as on my machine, you won’t see any window popping up.

Is this mandatory? because:

run(`script.bat`)

should just be equivalent on windows.
And this

start cmd @cmd /k echo "hello world"

seems also overly complex. Why not just

echo "hello world"

in the .bat?
Perhaps these are already artefacts of your search for a solution, so I ask explicitly.

I see why you do this… ok I will do and set it up here…

Works fine so far.
What is your exact completion criterion?
When I do this:

function test()
   run(`test.bat`)
   while ! isfile("test.out")
       sleep(5)
   end
end

test()

I get the cmd windows with a prompt (“hello world” at the beginning) and Julia prompt comes back when I say
touch "test.out"
in this CMD shell.

but doesn’t work with

run(`cmd start test.bat`)

… wait for it

Can you try instead of

run(`cmd start script.bat`)

this:

run(`cmd /C "start script.bat"`)

?

1 Like

Yes I understand your point, in reality that’s quite complex code that is run, the overly complex batch stuff is to get the stdout properly on separate windows in different cases.
run(script.bat) will put everything in the julia shell, which I don’t want. That won’t happen if script.bat is in turn running start cmd /k (I agree the @cmd was not necessary), but that is not always so, depending on cases[ic]. In short there are good reasons to have it like this :slight_smile:

Can you try instead of

run(`cmd start script.bat`)

this:

run(`cmd /C "start script.bat"`)

?

Thanks, sorry for lagging your posts all the time. I’ll try that.

I’m not really helping but can I comment that “system call” already has a specific meaning of calling into the kernel from user space, while what you are doing is running a user space program.

https://man7.org/linux/man-pages/man2/syscall.2.html

1 Like

Sorry for the terminology :slight_smile: I guess I’m running an external program by means of syscalls?

that does not seem to help. Actually I did have the /c, but after the start. None works. maybe I should be even more precise:
I used to use

run(program.exe inputfile.inp)
(btw sorry for not putting backticks around the command, how to differentiate it from preformatted text in the forum?)

but that put all of stdout in julia shell, while I wanted to have each instance of external program in a separate window to easily monitor its progress. that’s why I’m using

run(cmd start /c script.bat)

with script.bat running

start cmd @cmd /k program.exe inputfile.inp

This would be wrong, right is:

run(` cmd /c "start script.bat" `)

You can use triple back ticks:
```
code
```

I mean, nothing happens and it lands in a deadlock. When I abort, the external program corresponding to ic=1 is launched in a new window.

EDIT: I reached the limit of number of posts for a new user (I know I should have created an account way earlier). Any way to avoid waiting 22 hours?

I’ll keep on editing here. In the REPL, it fails to run without any message, stg I often get when Julia meets an unexpected error it cannot handle. EDIT in EDIT: sorry, that’s when I use the shortcut “execute code and move” in VSCode, if I use include(“mycode.jl”) it leads to a deadlock as in the shell, but I can’t abort without stopping the REPL so I don’t know if the external program would ever be run.

EDIT 2: script.bat is in the current folder, I’m using

joinpath(@__DIR__,"script.bat") 

to get its absolute path.

EDIT 3: I’m working on a MWE and will keep you updated!

without anything else is not possible, I need arguments in practice…

EDIT 4: Problem solved, but not fully understood.

First, I feel sorry to have forgotten an important detail: I was not using

run(command)

but

try @spawnat :any run(command) catch; run(command) end

using non-necessary multi-threading (remnant of an earlier version to provide parallel execution for julia -and not system- code).

The problem disappeared by removing the multi-threading stuff, but why??? I could not reproduce this on my MWE (that is similar to yours). There, both the single- and multi-threaded versions work. So I dug a bit further and yes, your intuition was right, in the end the simple

run(`cmd /c "start $myexe $myinput"`)

just works fine. Basically I had the /c on the wrong side of the start in the run statement, and had to compensate by using an extra script implementing yet another start. Still what multi-threading has to do with this is a mystery we don’t need to solve. Thanks for your time and wisdom!!!

What happens if you do just this in a REPL? Is it working than?

Is script.bat in your path or in the current folder?

I need to see your code.
I suspect, that you are creating your commands using Strings and than you probably do the

`...` 

or the

Cmd(`...`) 

wrong.
Please try to create a MWE which reproduces your issue without being overly complex. My guess is, while you do this you will solve your issue by your own :wink:
MWE = minimal working example (Please read: make it easier to help you)

And I am pretty sure, that this creates an issue.
What is done with that is:
Julia exec a cmd.exe with ARGV[1]="start" and ARGV[2]="script.bat".
So cmd.exe will start interactive, runs start without parameter which fails. Now cmd as an interactive shell stalls whatever is expected to run now.
I think it would be best to just use

run(`PATH\TO\script.bat`)

without anything else. Or if this is not possible:

run(`cmd /C "start script.bat"`)

The " are important here, because
ARGV[1] must be /C and
ARGV[2] must be the whole “start script.bat

You may be able to post again, thanx to @mbauman

2 Likes