I am writing a script which—at least in BASH—best works with subshells. The basic structure is “(do something; do something else; etc.) | the next thing|…”. A minimal working example (in BASH) would be:
(echo -n "Hello "; echo "world") > text.file
With Julia, the semicolon is a special character and must be escaped in a Cmd where it doesn’t perform as it would in bash so
pipeline(`echo -n "Hello "; echo "world"`,"text.file");
doesn’t work. My next thought was something like is shown in Pipelines in the manual.
run(pipeline(`echo -n "hello "` & `echo world`, "out.txt"));
The problem there is that on closer inspection to the text, the output is nondeterministic, which won’t work for me. I could not, however, get "worldhello "to print so I’m not sure if this is just an issue with the documentation.
I then just tried experimenting. Namely, I tried replacing the ampersand with a comma or semicolon. That just threw errors, which was what I expected. Next I tried making an array of commands.
cmdarray = [`echo -n "hello "` `echo world`]
run.(cmdarray) seems to work, but when I try
run.(pipeline.(cmdarray,"out.txt")) I end up with just “world” in the text file. Without the vectorizing
.'s, it fails with method errors, which is what I expected.
So I’m a bit lost as to what I can do.
edit: The example here may be a bit overly simple since you could print to the file and then print to it again, amending it. In my case, the data stream needs to sort of ship together or it’s a bit of a mess.
edit2: Another solution I thought up before bed is to generate the sequence as it would be used in bash and then use that.
function myStream(parameters) sequence = Some stuff generating a Cmd that leads to return `bash -c '$sequence'` end run(pipeline(myStream(parameters),`compressor`))
bash -c 'script' seems a bit janky and brute-force-ish. Another related option would be to generate the
$sequence by a Julia script that runs in a bash script, but I’d rather avoid two scripts.
edit3: Something else that popped into my head which I’ll try later today is to open the thing I’m passing the stream to, write it sequentially, then close it.
compressor = open(`the compressor I'm streaming to`,'a+') write(compressor,read(`one part of the sequential stream`)) write(compressor,read(`another part of the sequential stream`)) close(compressor)
I know with
'a+' I’m appending, but I’m doing it in a way that doesn’t close the stream so maybe that would work? I’m not sure. I’ll try this afternoon as well.