Using * as a wildcard in backtick commands

General Usage
1

Hi, I have seem to have a more intricate issue with backticks and string interpolation, and not sure what you mean by ‘command syntax’?

To compile several java .class files into one .jar files, I need to do:

run(`jar cf dest.jar folder/*.class`)

The issue is that the * character gets picked up by Julia and introduces single quotes, which in turn disrupts the jar process with an error:

failed process: Process(`jar cf dest.jar 'folder/*.class'`, ProcessExited(1))

However, by hand I am able to run without the awkward single quotes just fine.

jar cf dest.jar folder/*.class

The issue is that I need all the *.class files in one .jar file, which are generated from a Julia process. This automated single quoting seems a bit tricky in this situation… I think the documentation motivates the case of working around spaces in file names, but maybe a funny\ filename.txt would be an alternative solution rather than an automated 'funny filename.txt'?

Is there a workaround to avoid the introduction of single quotes? Thanks in advance.

2 Likes
2

You need to explicitly expand globbing using the Glob.jl package.

2 Likes
3

To expand on that, the reason this doesn’t work is that expanding * to match any file in the directory is something that your shell (generally bash) does for you. Julia commands are not run by the shell at all, but are just executed directly (see https://docs.julialang.org/en/stable/manual/running-external-programs/#Running-External-Programs-1 , specifically the third bullet point).

This also explains why bash aliases and shell functions can’t be used inside commands (unless your command itself launches bash and passes some string to bash to execute). See also Success() does not give same result as running in terminal and Getting Unwanted Quotes in System Calls

1 Like
4

There’s a plan to add more shell-like features to backticks in the future, which is why we’ve deprecated unquoted/unescaped shell metacharacters in backticks in 0.6. In the future this kind of globbing may be built in, and Julia will effectively implement a basic portable shell in backticks, including pipelines, I/O redirection and probably globbing and home directory expansion. It would also allow cool things like splicing a Julia task into a pipeline or reading to/from arbitrary Julia I/O objects. I was going to try to do this for 1.0 but realized that it was a ton of work and can be added in 1.x now that we’ve deprecated shell metacharacters.

4 Likes
5

Some time ago, I wanted shell-like functionality for Julia’s backtick commands.
So I wrote a small convenience module to help.
It uses command literals.

run(sh`jar cf dest.jar folder/*.class`)

Also supports cmd and ps (powershell) on Windows.

Julia-0.6.0> sh`jar cf dest.jar folder/*.class`
`sh -c 'jar cf dest.jar folder/*.class'`

Julia-0.6.0> cmd`jar cf dest.jar folder/*.class`
`cmd /s /c '"jar cf dest.jar folder/*.class"'`

Julia-0.6.0> ps`jar cf dest.jar folder/*.class`
`powershell -Command 'jar cf dest.jar folder/*.class'`

And if anyone is interested, here’s the module:

__precompile__()

module ShellCommands

export @cmd_cmd, @ps_cmd, @busybox_cmd, @sh_cmd

macro cmd_cmd(s_str)
    s_expr = parse(string('"', escape_string(s_str), '"'))
    return esc(:(Cmd(Base.cmd_gen(("cmd", "/s", "/c", string('"', $s_expr, '"'))), windows_verbatim=true)))
end

macro ps_cmd(s_str)
    s_expr = parse(string('"', escape_string(s_str), '"'))
    return esc(:(Base.cmd_gen(("powershell", "-Command", $s_expr))))
end

macro busybox_cmd(s_str)
    s_expr = parse(string('"', escape_string(s_str), '"'))
    return esc(:(Base.cmd_gen(("busybox", "sh", "-c", $s_expr))))
end

macro sh_cmd(s_str)
    s_expr = parse(string('"', escape_string(s_str), '"'))
    return esc(:(Base.cmd_gen(("sh", "-c", $s_expr))))
end

end
7 Likes
6

Thanks a lot, just came across this! A v1.x follow up for posterity: the parse commands will be replaced with Meta.parse