Understanding Shell Interactions within Julia scripts

General Usage
1

Hello!

I am a PhD. student hoping to use Julia for some extensive ML CFD data analysis. I am having trouble with shell scope within a Julia script. I need to source (via Linux bash shell) a bashrc file for OpenFOAM (CFD software) so that I can run several commands that come from said sourcing.

From my understanding, the “run” command in Julia runs a bare bones shell (a shell without bashrc or user-specific aliases). When I run the command:
run(source /home/matt/OpenFOAM_Source_Code/foam-extend-4.1/foam-extend-4.1/etc/bashrc)
the output is:
ERROR: IOError: could not spawn source /home/matt/OpenFOAM_Source_Code/foam-extend-4.1/foam-extend-4.1/etc/bashrc: no such file or directory (ENOENT)

However, when this command is run with the “;” operator I do not get an error. Additionally, running "run(`cat /home/matt/OpenFOAM_Source_Code/foam-extend-4.1/foam-extend-4.1/etc/bashrc’) successfully reproduces the text in the bashrc file. Yet, this shell instance does not seem to be persistent because when I run any command that source the bashrc should give me, I just get the standard shell “command not found” error. I have tried searching both shell persistence and shell characteristics for Julia but have yet to find a solution to this problem.

So my question is, would I be better off with a bash script that runs all the shell commands I need or is their a work-around for this behavior where I can run these commands in Julia.

I already understand that I could do one of two things:

  1. Build a text shell file that has the needed sourcing, along with the command I need ran. However, I will be running several hundreds of such commands, so re-sourcing every time will probably led to performance issues.

  2. Use pure bash for handling files and running the OpenFOAM cases, and just post-process the data in Julia.

I would like to learn a little more about Julia and thus I am attempting to do this with mostly pure Julia (with the run commands).

Thank you in advance for reading such a long question and time spend diagnosing this code.

I am using Julia Version 1.6.2 (2021-07-14) along with WSL Ubuntu 20.04.

2

this should clear much confusion:

there’s no Shell interaction because Julia doesn’t execute binaries via shell, so for example, * doesn’t get expanded:

julia> readlines(`pwd`)
1-element Vector{String}:
 "/"

julia> readlines(`ls`)
24-element Vector{String}:
 "bin"
 "boot"
 "desktopfs-pkgs.txt"
 "dev"
 "etc"
 "home"
 "lib"
 "lib64"
...
julia> split(read(`echo *`, String))
ERROR: LoadError: parsing command `echo *`: special characters "#{}()[]<>|&*?~;" must be quoted in commands
Stacktrace:
 [1] error(s::String)

julia> split(read(`sh -c 'echo *'`, String))
24-element Vector{SubString{String}}:
 "bin"
 "boot"
 "desktopfs-pkgs.txt"
 "dev"
 "etc"
 "home"
 "lib"
1 Like
3

Yes, this makes sense. Re-reading the shell page for Julia I see it parses and then uses the exec command. Thus I believe it is necessary to source every time. Thus for my problem the correct syntax would be:

run(`bash -c 'source $HOME/OpenFOAM_Source_Code/foam-extend-4.1/foam-extend-4.1/etc/bashrc ; blockMesh -help'`)

which I could just make a function to just change the last command to what I need. This appears to do exactly what I need without having to create cumbersome bash text files on-the-fly for every command. Thanks so much!