# Better support for running external commands

**URL:** <https://discourse.julialang.org/t/better-support-for-running-external-commands/44933>\
**Category:** New to Julia\
**Tags:** run, cmd\
**Created:** [August 14, 2020, 2:18pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933 "2020-08-14T14:18:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 2:18pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/1 "2020-08-14T14:18:46Z")

</div>

Currently, in Julia the `run` function is not as stable as the rest of the Julia.

You can’t run external commands with special characters. The different behavior of `Cmd` from `String` makes using it inconvenient. Windows support is not like other operating systems.

Running something as simple as the following in Julia requires a lot of workarounds.

```python
# python
import os
os.system("echo hey/*")
os.system("echo hey && echo bye")
x="echo bye"
os.system(x)

```

Trying these in Julia:

1. Running Commands with special characters:

using `Cmd` direcrtly fails

```julia
julia> run(`echo hey/*`)
ERROR: LoadError: parsing command `echo hey/*`: special characters "#{}()[]<>|&*?~;" must be quoted in commands
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] shell_parse(::String, ::Bool; special::String) at .\shell.jl:100
 [3] @cmd(::LineNumberNode, ::Module, ::Any) at .\cmd.jl:389
in expression starting at REPL[3]:1

```

Using `@cmd` fails

```julia
julia> @cmd run"echo hey/*"
ERROR: LoadError: MethodError: no method matching shell_parse(::Expr; special="#{}()[]<>|&*?~;")
Closest candidates are:
  shell_parse(::AbstractString) at shell.jl:18 got unsupported keyword argument "special"
  shell_parse(::AbstractString, ::Bool; special) at shell.jl:18
Stacktrace:
 [1] @cmd(::LineNumberNode, ::Module, ::Any) at .\cmd.jl:389
in expression starting at REPL[6]:1

```

There is no `@raw_cmd` and `@raw_string` doesn’t work as stated in 3

```julia
julia> run(raw`echo hey/*`)
ERROR: LoadError: UndefVarError: @raw_cmd not defined
in expression starting at REPL[1]:1

julia> run(raw"echo hey/*")
ERROR: MethodError: no method matching run(::String)
Closest candidates are:
  run(::Base.AbstractCmd, ::Any...; wait) at process.jl:437
Stacktrace:
 [1] top-level scope at REPL[2]:1

julia> run(`$(raw"echo hey/*")`)
ERROR: IOError: could not spawn `'echo hey/*'`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #585 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at .\process.jl:439
 [6] run(::Cmd) at .\process.jl:438
 [7] top-level scope at REPL[1]:1

```

In some cases, it is possible to explicitly escape the special characters but not always

```julia
# this works
julia> run(`echo hey/\*`)
hey/*
Process(`echo 'hey/*'`, ProcessExited(0))

# this doesn't
julia> run(`echo hi \&\& echo bye`) # or run(`echo hi '&&' echo bye`)
hi && echo bye
Process(`echo hi '&&' echo bye`, ProcessExited(0))

```

1. Out of the box Windows Support

This requires another hackery if you want to run this on Windows:

```julia
# happens in pwsh, powershell, cmd
julia> run(`echo hey/\*`)
ERROR: IOError: could not spawn `echo 'hey/*'`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #585 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at .\process.jl:439
 [6] run(::Cmd) at .\process.jl:438
 [7] top-level scope at REPL[1]:1

```

```julia
julia> run(`cmd /c echo hey/\*`)
hey/*
Process(`cmd /c echo 'hey/*'`, ProcessExited(0))

```

1. Possibility of using `String` for building a command

```julia
julia> x = "echo bye"
"echo bye"

```

Using `String` directly fails

```julia
julia> Cmd(x)
ERROR: MethodError: no method matching Cmd(::String)
Closest candidates are:
  Cmd(::Array{String,1}) at cmd.jl:16
  Cmd(::Cmd; ignorestatus, env, dir, detach, windows_verbatim, windows_hide) at cmd.jl:21
  Cmd(::Cmd, ::Any, ::Any, ::Any, ::Any) at cmd.jl:18
Stacktrace:
 [1] top-level scope at REPL[40]:1

```

Trying interpolation

```julia
julia> run(`$x`)
ERROR: IOError: could not spawn `'echo bye'`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #585 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at .\process.jl:439
 [6] run(::Cmd) at .\process.jl:438
 [7] top-level scope at REPL[31]:1

```

Trying `@cmd`

```julia
julia> @cmd($x)
ERROR: LoadError: MethodError: no method matching shell_parse(::Expr; special="#{}()[]<>|&*?~;")
Closest candidates are:
  shell_parse(::AbstractString) at shell.jl:18 got unsupported keyword argument "special"
  shell_parse(::AbstractString, ::Bool; special) at shell.jl:18
Stacktrace:
 [1] @cmd(::LineNumberNode, ::Module, ::Any) at .\cmd.jl:389
in expression starting at REPL[36]:1

```

```julia
julia> @cmd(x)
ERROR: LoadError: MethodError: no method matching shell_parse(::Symbol; special="#{}()[]<>|&*?~;")
Closest candidates are:
  shell_parse(::AbstractString) at shell.jl:18 got unsupported keyword argument "special"
  shell_parse(::AbstractString, ::Bool; special) at shell.jl:18
Stacktrace:
 [1] @cmd(::LineNumberNode, ::Module, ::Any) at .\cmd.jl:389
in expression starting at REPL[37]:1

```

```julia
julia> @cmd("$x")
ERROR: LoadError: MethodError: no method matching shell_parse(::Expr; special="#{}()[]<>|&*?~;")
Closest candidates are:
  shell_parse(::AbstractString) at shell.jl:18 got unsupported keyword argument "special"
  shell_parse(::AbstractString, ::Bool; special) at shell.jl:18
Stacktrace:
 [1] @cmd(::LineNumberNode, ::Module, ::Any) at .\cmd.jl:389
in expression starting at REPL[39]:1

```

My suggestion for 1 and 3: allow using `String` with `run`. This will fix 1 with the help of `@raw_string`. This will also fix 3 by directly sending the command to the system without putting `'` around it.  
My suggestion for 2: detect the host shell and adjust `run` accordignly. If not possible define a `run2` which automatically finds `pwsh`, `powershell`, or `cmd`. In Julia 2, make `run2` the default `run`.

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [August 14, 2020, 3:01pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/2 "2020-08-14T15:01:17Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> You can’t run external commands with special characters.

You can do this by quoting the command with single or double backticks.

> [@Amin\_Yahyaabadi](#):
>
> using `Cmd` direcrtly fails
> 
> ```julia
> julia
> 
> ```

This is because globs (e.g. `*` to match any string) don’t work in Julia (yet).

> [@Amin\_Yahyaabadi](#):
>
> ```julia
> # this works
> julia> run(`echo hey/\*`)
> hey/*
> Process(`echo 'hey/*'`, ProcessExited(0))
> 
> ```

This only matches a path with the literal name `*`, which is probably not what you want.

> [@Amin\_Yahyaabadi](#):
>
> ```julia
> # this doesn't
> julia> run(`echo hi \&\& echo bye`) # or run(`echo hi '&&' echo bye`)
> hi && echo bye
> Process(`echo hi '&&' echo bye`, ProcessExited(0))
> 
> ```

It does what you ask it to, it escapes `&&` as an argument to echo. `&&` for command chaining, like in bash, is also not currently supported.

> [@Amin\_Yahyaabadi](#):
>
> 1. Possibility of using `String` for building a command
> 
> ```julia
> julia> x = "echo bye"
> "echo bye"
> 
> ```

This is actually a feature, because it protects you from command injection. You usually don’t just want to run arbitrary strings as commands. If you really want to create a command from a string like this, you need to call `Base.shell_parse` directly, but this could definitely be documented better.

I do agree that there are still a lot of ways running external commands could be improved. To me, that would mean moving [Glob.jl](https://github.com/vtjnash/Glob.jl) into base, supporting piping and chaining constructs like `|`, `>`, `&&`, etc. directly in command literals, and I’d also really like to see [parse command interpolations at parse time · Issue #3150 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/3150) being addressed.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 14, 2020, 3:10pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/3 "2020-08-14T15:10:53Z")

</div>

Note that all these features you’re talking about (globbing, piping, short circuiting) are shell’s features, julia doesn’t use the shell to parse the command, which is arguably a feature as it makes the code independent from the currently used shell

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [August 14, 2020, 3:17pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/4 "2020-08-14T15:17:13Z")

</div>

Note however that this doesn’t mean, we can’t have those features in Julia eventually. Command literals are already their own small shell language and could certainly support these features, independent of your system’s shell.

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 3:20pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/5 "2020-08-14T15:20:43Z")

</div>

> [@simeonschaub](#):
>
> You can do this by quoting the command with single or double backticks

In some cases (like this simple `echo hey/*`) this is possible, but this does not work all the time. `&&` was an example. Additionally, going through each command one by one and escaping them is very inconvenient.

> [@simeonschaub](#):
>
> You usually don’t just want to run arbitrary strings as commands.

Wanting to run a command using a simple `String` is not a rare request. Among the languages I know, only PowerShell and Julia have weird behaviors for running external commands. PowerShell keeps introducing workarounds for its current broken behavior. Julia tries to stick to the API, but `run` still does not work properly.

> [@simeonschaub](#):
>
> Base.shell\_parse

As I showed, `Base.shell_parse` is not stable and a lot of the methods are not defined.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 14, 2020, 3:27pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/6 "2020-08-14T15:27:34Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> Wanting to run a command using a simple `String` is not a rare request.

It may not be, but as explained by @simeonschaub and @giordano above, it is not a well-defined request either, as it depends on shell semantics, which are underspecified (your shell could be literally anything).

If you don’t care about the potential issues, you can always just do something like

```julia
bashit(str) = run(`bash -c $str`)

bashit("echo hey && echo bye")

```

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 3:57pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/7 "2020-08-14T15:57:22Z")

</div>

> [@Tamas\_Papp](#):
>
> If you don’t care about the potential issues, you can always just do something like

I understand that this could be feature for those who are interested but the issues that the current behavior brings are more than what a simple `run(x::String)` has.

For some context, the most used languages don’t have such behavior, and I have not faced such issues in those languages.

```js
# js
const childProcess = require('child_process');
childProcess.execSync("echo hi/* && echo bye")

```

```python
# python
import os
os.system("echo hi/* && echo bye")

```

```c++
# c++
#include<stdlib.h>

int main() {
   std::system("echo hi/* && echo bye");
   return 0;
}

```

Probably I can continue with the examples, but you get the idea.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [August 14, 2020, 5:03pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/8 "2020-08-14T17:03:18Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> For some context, the most used languages don’t have such behavior, and I have not faced such issues in those languages.

The `system` command in other languages executes a shell. `run` in Julia is not a shell — it is a function to directly launch processes (analogous to [`subprocess.call` in Python](https://docs.python.org/2/library/subprocess.html#module-subprocess)). By relying on the shell to launch processes you run into lots of other issues (+ portability problems): [Shelling Out Sucks](https://julialang.org/blog/2012/03/shelling-out-sucks/) … the Python developers also concluded the same thing ([PEP 324](https://www.python.org/dev/peps/pep-0324/#motivation)).

Julia _has_ globbing and pipelines etcetera, but they aren’t implemented with shell _syntax_. They are implemented in Julia, e.g. pipelines via `pipeline(...)` and globbing via [Glob.jl](https://github.com/vtjnash/Glob.jl).

Just don’t expect `run` to be equivalent to `bash`. If you want to launch a shell like `bash` and pass it a string to execute, you can do that in Julia, of course (by spawning the shell explicitly).

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 8:09pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/9 "2020-08-14T20:09:37Z")

</div>

I wish “being explicit” was enough, but it does not work always. I hope someday `Cmd` works like a `String` so I can use it without applying all these workarounds.

Here is another case that `Cmd` is broken:

Consider that I want to **explicitly** run this Julia command:

```julia
julia> ex = :( using Printf; @printf("%f",1) )
quote
    using Printf
    #= REPL[31]:1 =#
    #= REPL[31]:1 =# @printf "%f" 1
end

```

First try

```julia
julia> run(`julia -e "$ex"`)
ERROR: LoadError: UndefVarError: @printf not defined
in expression starting at none:4
ERROR: failed process: Process(`julia -e 'begin
    using Printf
    #= REPL[31]:1 =#
    #= REPL[31]:1 =# @printf "%f" 1
end'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run(::Cmd) at .\process.jl:438
 [4] top-level scope at REPL[32]:1

```

Another try

```julia
julia> run(`julia -e $ex`)
ERROR: LoadError: UndefVarError: @printf not defined
in expression starting at none:4
ERROR: failed process: Process(`julia -e 'begin
    using Printf
    #= REPL[31]:1 =#
    #= REPL[31]:1 =# @printf "%f" 1
end'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run(::Cmd) at .\process.jl:438
 [4] top-level scope at REPL[35]:1

```

Trying to hack into Julia AST to make this work!

```julia
julia> ex = Expr(:toplevel, :(using Printf), quote
           @printf("%f",1)
       end)
:($(Expr(:toplevel, :(using Printf), quote
    #= REPL[38]:2 =#
    #= REPL[38]:2 =# @printf "%f" 1
end)))

```

```julia
julia> run(`julia -e $ex`) # same as run(`julia -e "$ex"`)
ERROR: syntax: "$" expression outside quote
Stacktrace:
 [1] top-level scope at none:1
ERROR: failed process: Process(`julia -e '$(Expr(:toplevel, :(using Printf), quote
    #= REPL[38]:2 =#
    #= REPL[38]:2 =# @printf "%f" 1
end))'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run(::Cmd) at .\process.jl:438
 [4] top-level scope at REPL[39]:1

```

Just give up and introduce another workaround:

```julia
julia> Cmd_fixer(expr::Expr) = replace(string(expr), r"^begin([\s\S]*)end$"=>s"\1");

```

```julia
julia> ex = :( using Printf; @printf("%f",1) );

julia> run(`julia -e $(Cmd_fixer(ex) )`)
1.000000Process(`julia -e '
    using Printf
    #= REPL[45]:1 =#
    #= REPL[45]:1 =# @printf "%f" 1
'`, ProcessExited(0))

```

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [August 14, 2020, 9:03pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/10 "2020-08-14T21:03:42Z")

</div>

@Amin_Yahyaabadi - I don’t understand the actual problem here. Is there a reason why you are receiving julia commands and not strings which you want to toss into a REPL? It seems a little tricky to do what you’re doing.  
while the following:

```julia
ex = "using Printf; @printf(\"%f\",1)"
"using Printf; @printf(\"%f\",1)"

julia> run(`julia -e $ex`)
1.000000Process(`julia -e 'using Printf; @printf("%f",1)'`, ProcessExited(0))

```

is really easy :).

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 9:28pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/11 "2020-08-14T21:28:10Z")

</div>

Calling external commands is very common in general. Here, I was just showing a case that specifying the program explicitly does not help.

If your question is about the exact application of this, you would do this when you want to run a command outside of your current session. For example, I used external Julia processes in [SnoopCompileBot](https://github.com/aminya/SnoopCompileBot.jl) to snoop the compilation of programs.

About your solution, what if you could not escape the strings manually? Taking your solution with @Tamas_Papp’s suggestion:

```julia
julia> ex= raw"""using Printf; @printf("%f",1)"""
"using Printf; @printf(\"%f\",1)"

julia> bashit(str) = run(`bash -c $str`);

julia> bashit("julia -e $ex")
/bin/bash: -c: line 0: syntax error near unexpected token `"%f",1'
/bin/bash: -c: line 0: `julia -e using Printf; @printf("%f",1)'
ERROR: failed process: Process(`bash -c 'julia -e using Printf; @printf("%f",1)'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run at .\process.jl:438 [inlined]
 [4] bashit(::String) at .\REPL[12]:1
 [5] top-level scope at REPL[13]:1

julia> bashit("julia -e '$ex'")
/bin/bash: julia: command not found
ERROR: failed process: Process(`bash -c "julia -e 'using Printf; @printf(\"%f\",1)'"`, ProcessExited(127)) [127]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run at .\process.jl:438 [inlined]
 [4] bashit(::String) at .\REPL[2]:1
 [5] top-level scope at REPL[3]:1

```

Try cmd:

```julia
julia> run(`cmd /c 'julia -e $ex'`)
ERROR: syntax: "$" expression outside quote
Stacktrace:
 [1] top-level scope at none:1
ERROR: failed process: Process(`cmd /c 'julia -e $ex'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run(::Cmd) at .\process.jl:438

julia> run(`cmd /c "julia -e $ex"`)
ERROR: syntax: incomplete: premature end of input
Stacktrace:
 [1] top-level scope at none:1
ERROR: failed process: Process(`cmd /c 'julia -e using Printf; @printf("%f",1)'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run(::Cmd) at .\process.jl:438
 [4] top-level scope at REPL[3]:1

```

Trying some Nodejs code:

```julia
julia> ex= raw"""node -e 'console.log("x")' """
"node -e 'console.log(\"x\")' "

```

using bashit

```julia
julia> bashit(str) = run(`bash -c $ex`);

julia> bashit(ex)
/bin/bash: node: command not found
ERROR: failed process: Process(`bash -c "node -e 'console.log(\"x\")' "`, ProcessExited(127)) [127]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run at .\process.jl:438 [inlined]
 [4] bashit(::String) at .\REPL[13]:1
 [5] top-level scope at REPL[14]:1

julia> run(`node -v`) # but node is there
v14.7.0
Process(`node -v`, ProcessExited(0))

```

Try cmd

```julia
julia> run(`cmd /c "$ex"`) # x is not printed
Process(`cmd /c "node -e 'console.log(\"x\")' "`, ProcessExited(0))

```

The point is that for each situation a different workaround is required and. Keeping track of these in a real world application, which you need to run multiple external commands, this becomes frustrating.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 14, 2020, 10:00pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/12 "2020-08-14T22:00:30Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> The point is that for each situation a different workaround is required

What I see here is that having to deal with each underlying shell is a pain. Which is the whole point of not relying on it.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [August 14, 2020, 10:00pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/13 "2020-08-14T22:00:57Z")

</div>

i see what you’re saying now. Normally what I would do in these instances is, write to disk and run the code. But I think I’ve only done that once. Not sure what the generic solution is. Just trying to understand the problem.

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 14, 2020, 10:21pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/14 "2020-08-14T22:21:49Z")

</div>

Yes. I ended up deprecating macros in SnoopCompileBot and only accept the code that is written to the disk. However, I still use `Cmd_fixer` for interpolating my code into the external Julia process.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 15, 2020, 7:48am UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/15 "2020-08-15T07:48:39Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> Currently, in Julia the `run` function is not as stable as the rest of the Julia.

Did you read [Running External Programs · The Julia Language](https://docs.julialang.org/en/v1/manual/running-external-programs/#Running-External-Programs-1). It seems a lot of the issues here are due to a misunderstanding of what `run` actually does. The docs explicitly discusses some of the issues you encounter here.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 15, 2020, 7:57am UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/16 "2020-08-15T07:57:43Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> using bashit
> 
> ```julia
> julia> bashit(str) = run(`bash -c $ex`);
> 
> julia> bashit(ex)
> /bin/bash: node: command not found
> 
> ```

Note that this function is just a suggestion, not a well-tested and robust implementation of anything.

It is still not clear to me what you want Julia to do here — what you call “external commands” is actually _code_ written in some shell syntax. You did not say which shell you want, so I imagine you are assuming they are the same, but in fact even simple globbing has different behavior in various shells in some corner cases.

A shell is a special kind of programming language, `run` does not anything like that. But most of that functionality is available in other ways in Julia. Instead of invoking the shell, I would recommend just coding a solution in Julia.

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 15, 2020, 10:26pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/17 "2020-08-15T22:26:30Z")

</div>

Yes, and many of the examples don’t work on Windows unless you modify them. There is no note on how to run these commands on **any** operating system. The examples use things that are not installed on every operating system: `grep, perl, rm -rf, cut -d:`. As soon as you start to add your `cmd /c` or `bash -c` modification, you run into all of the errors I mentioned, and you should apply special workarounds for them.

[https://docs.julialang.org/en/v1/manual/running-external-programs/#Running-External-Programs-1](https://docs.julialang.org/en/v1/manual/running-external-programs/#Running-External-Programs-1)

```julia
julia> run(`echo hello`);
ERROR: IOError: could not spawn `echo hello`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #585 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at .\process.jl:439
 [6] run(::Cmd) at .\process.jl:438
 [7] top-level scope at REPL[7]:1

```

Correct form

```julia
julia> run(`cmd /c echo hello`);
hello

```

[https://docs.julialang.org/en/v1/manual/running-external-programs/#Pipelines-1](https://docs.julialang.org/en/v1/manual/running-external-programs/#Pipelines-1)

```julia
julia> run(`echo hello` & `echo world`);
ERROR: IOError: could not spawn `echo hello`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] _spawn(::Cmd, ::Array{Any,1}, ::Base.ProcessChain) at .\process.jl:181
 [3] _spawn(::Base.AndCmds, ::Array{Any,1}, ::Base.ProcessChain) at .\process.jl:174
 [4] #587 at .\process.jl:120 [inlined]
 [5] setup_stdios(::Base.var"#587#588"{Base.AndCmds}, ::Array{Any,1}) at .\process.jl:196
 [6] _spawn at .\process.jl:119 [inlined]
 [7] run(::Base.AndCmds; wait::Bool) at .\process.jl:439
 [8] run(::Base.AndCmds) at .\process.jl:438
 [9] top-level scope at REPL[3]:1

```

Correct form:

```julia
julia> run(`cmd /c echo hello` & `cmd /c echo world`);
hello
world

```

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 15, 2020, 10:36pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/18 "2020-08-15T22:36:38Z")

</div>

> [@Tamas\_Papp](#):
>
> A shell is a special kind of programming language, `run` does not anything like that. But most of that functionality is available in other ways in Julia. Instead of invoking the shell, I would recommend just coding a solution in Julia.

[That’s the best practice](https://github.com/aminya/SnoopCompileBot.jl/blob/2bca967ae18c57aacb2028e0a550262c9007befa/src/postprocess.jl#L6), however, it is inevitable in a real application. Anything that is considered in the `General Programming` area needs this external interface. For example, to check if your file is tracked using `git` you would do something like this:

Here, I should try three different solutions until one of them work:

```julia
julia> ex = raw""" git ls-files 'Project.toml' | echo """
" git ls-files 'Project.toml' | echo "

julia> run(pipeline(`basch -c $ex`))
ERROR: IOError: could not spawn `basch -c " git ls-files 'Project.toml' | echo "`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #585 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at .\process.jl:439
 [6] run(::Cmd) at .\process.jl:438
 [7] top-level scope at REPL[16]:1

julia> run(`bash -c $ex`)

Process(`bash -c " git ls-files 'Project.toml' | echo "`, ProcessExited(0))

julia> run(pipeline(`pwsh -c $ex`))
Project.toml
Process(`pwsh -c " git ls-files 'Project.toml' | echo "`, ProcessExited(0))

```

Probably I can try to make [Git.jl](https://github.com/JuliaPackaging/Git.jl) compatible with newer Julia, but that is too much work for a simple check.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 15, 2020, 10:47pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/19 "2020-08-15T22:47:14Z")

</div>

> [@Amin\_Yahyaabadi](#):
>
> ```julia
> ex = raw""" git ls-files 'Project.toml' | echo """
> " git ls-files 'Project.toml' | echo "
> 
> ```

You had a typo in your first command. This works:

```julia
julia> ex = raw""" git ls-files 'Project.toml' """                                                           
" git ls-files 'Project.toml' "                                                                              
                                                                                                               
julia> run(pipeline(`bash -c "$(ex)"`))                                                                        
Project.toml                                                                                                   
Process(`bash -c " git ls-files 'Project.toml' "`, ProcessExited(0))   

```

---

<div class="post-metadata">

**Author:** ![Amin\_Yahyaabadi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amin_yahyaabadi/32/9826_2.png) [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Post date:** [August 15, 2020, 10:50pm UTC](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933/20 "2020-08-15T22:50:49Z")

</div>

Yes, sorry. I was trying to make the command runnable on different operating systems (because of `grep`, so people can try it without issues. This was the [original command](https://github.com/aminya/SnoopCompileBot.jl/commit/4bb081093f6a552b5e4d7efa81dd197a23869225#diff-493d89206c11fea99259d0771034d26aL172):

```julia
# this checks if Manifest is tracked by Git, and if so it reverts it to the final state
git ls-files 'Manifest.toml' | grep . && git checkout -- 'Manifest.toml'

```

I finally gave up trying:  
[https://github.com/aminya/SnoopCompileBot.jl/blob/2bca967ae18c57aacb2028e0a550262c9007befa/src/postprocess.jl#L21](https://github.com/aminya/SnoopCompileBot.jl/blob/2bca967ae18c57aacb2028e0a550262c9007befa/src/postprocess.jl#L21)

[Next page](https://discourse.julialang.org/t/better-support-for-running-external-commands/44933.md?page=2)