# Unzipping single file with a shell command
**URL:** https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263
**Category:** General Usage
**Created:** [April 18, 2019, 7:24am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263 "2019-04-18T07:24:48Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Jeremy](https://discourse.julialang.org/u/Jeremy)
#### Post date: [April 18, 2019, 7:24am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/1 "2019-04-18T07:24:48Z")
I’m trying to update my package, ClimateDataIO, for Julia 0.7 and beyond but because of a problem with some function down the chain in ZipFile/CSV/etc. I’ve had to switch to a shell command to unzip the file that I want. The following command works perfectly in the command line bug none of the many forms I’ve tried work in the REPL.
```julia
unzip -p /home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data
```
My last attempt looked like this:
```julia
julia> read(`unzip -p '/home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data'`,String)
ERROR: failed process: Process(`unzip -p '/home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data'`, ProcessExited(9)) [9]
Stacktrace:
[1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:42
[2] pipeline_error at ./process.jl:712 [inlined]
[3] read(::Cmd) at ./process.jl:648
[4] read(::Cmd, ::Type{String}) at ./process.jl:652
[5] top-level scope at none:0
```
`run` gives the same result.
```julia
julia> run(`unzip -p '/home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data'`)
ERROR: failed process: Process(`unzip -p '/home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data'`, ProcessExited(9)) [9]
Stacktrace:
[1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:42
[2] pipeline_error at ./process.jl:712 [inlined]
[3] #run#509(::Bool, ::Function, ::Cmd) at ./process.jl:670
[4] run(::Cmd) at ./process.jl:668
[5] top-level scope at none:0
```
What am I doing wrong?
---
### Author:  [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [April 18, 2019, 7:36am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/2 "2019-04-18T07:36:16Z")
You are probably looking for `pipeline`. From the docs (`?pipeline` in the REPL):
```julia
pipeline(from, to, ...)
Create a pipeline from a data source to a destination. The source and
destination can be commands, I/O streams, strings, or results of other
pipeline calls. At least one argument must be a command. Strings
refer to filenames. When called with more than two arguments, they
are chained together from left to right. For example, pipeline(a,b,c)
is equivalent to pipeline(pipeline(a,b),c). This provides a more
concise way to specify multi-stage pipelines.
Examples:
run(pipeline(`ls`, `grep xyz`))
run(pipeline(`ls`, "out.txt"))
run(pipeline("out.txt", `grep xyz`))
```
---
### Author:  [@Jeremy](https://discourse.julialang.org/u/Jeremy)
#### Post date: [April 18, 2019, 7:43am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/3 "2019-04-18T07:43:14Z")
I tried that as well but without any success. It complains about the file not existing even though doing it in the shell works. In this case it complains about “No such file or directory”.
```julia
julia> run(pipeline("/home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data >/tmp/2016-12-11T203000_AIU-1359.data",`unzip -p`))
ERROR: IOError: open: no such file or directory (ENOENT)
Stacktrace:
[1] uv_error at ./libuv.jl:85 [inlined]
[2] open(::String, ::UInt8, ::UInt16) at ./filesystem.jl:81
[3] setup_stdio(::Base.FileRedirect, ::Bool) at ./process.jl:472
[4] setup_stdio(::getfield(Base, Symbol("##499#500")){Cmd}, ::Tuple{Base.FileRedirect,RawFD,RawFD}) at ./process.jl:487
[5] #_spawn#498(::Nothing, ::Function, ::Cmd, ::Tuple{Base.FileRedirect,RawFD,RawFD}) at ./process.jl:511
[6] (::getfield(Base, Symbol("#kw##_spawn")))(::NamedTuple{(:chain,),Tuple{Nothing}}, ::typeof(Base._spawn), ::Cmd, ::Tuple{Base.FileRedirect,RawFD,RawFD}) at ./none:0
[7] #_spawn#495(::Nothing, ::Function, ::Base.CmdRedirect, ::Tuple{RawFD,RawFD,RawFD}) at ./process.jl:401
[8] _spawn at ./process.jl:401 [inlined]
[9] #run#509(::Bool, ::Function, ::Base.CmdRedirect) at ./process.jl:669
[10] run(::Base.CmdRedirect) at ./process.jl:668
[11] top-level scope at none:0
```
Even if I create the file, `touch("/tmp/2016-12-11T203000_AIU-1359.data")`, it gives the same error when I execute the `run` command.
---
### Author:  [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [April 18, 2019, 7:45am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/4 "2019-04-18T07:45:52Z")
Did you read the docs I linked? They say:
```julia
run(pipeline(`unzip -p /path/to/file`, "path/to/outputfile"))
```
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [April 18, 2019, 7:50am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/5 "2019-04-18T07:50:55Z")
Can you tell us more about your situation? I cannot reproduce, i.e. `read(`unzip -p '/path/to/file/foo.zip'`, String)` works perfectly for me.
On the other hand, `read(`unzip -p '/path/to/file/foo.zip /path/to/second/file/foo2.zip'`, String)` gives the same `ERROR: failed process: Process(`unzip -p '/path/to/file/foo.zip /path/to/second/file/foo2.zip'`, ProcessExited(9)) [9]`
So I’d guess that `unzip -p` does not like multiple files as input (and your command hat multiple input files if I read correctly). Can’t you just read one after the other?
Edit: ok, my suggestions where not helpful. preserved below.
I’d use e.g.
```julia
julia> f=open(`cat ./foo.c`, "r");
julia> s=read(f.out, String)
"#include \n..."
julia> close(f)
```
If you open a command with `"r+"`, then you can also write to `f.in` and close it, in order to interact with your command (relevant if you want to pass things into your command’s stdin). Be warned: writes to pipes are by default unbuffered. This is abysmally slow for some workloads (every single write gives a bunch of syscalls; check `strace` to see whether you need buffering tricks).
---
### Author:  [@Jeremy](https://discourse.julialang.org/u/Jeremy)
#### Post date: [April 18, 2019, 7:59am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/6 "2019-04-18T07:59:56Z")
Yup, long before my first post along with numerous iterations and combinations of `run`, `read`, and `pipeline`. The last iteration did work so thanks for getting me to try it yet again. I really could have sworn that I had tried this combination.
🤷♂️ C’est la vie.
This is the working command.
```julia
run(pipeline(`unzip -p /home/user/.julia/packages/ClimateDataIO/XleN/test/2016-12-11T203000_AIU-1359.ghg 2016-12-11T203000_AIU-1359.data`,"/tmp/2016-12-11T203000_AIU-1359.data"))
```
---
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [April 18, 2019, 9:49am UTC](https://discourse.julialang.org/t/unzipping-single-file-with-a-shell-command/23263/7 "2019-04-18T09:49:26Z")
DataDeps.jl includes `unpack` as one
of its helpers
see the code in
[https://github.com/oxinabox/DataDeps.jl/blob/c5256065e4396a49e43c680e35e68357a6eb01f3/src/post\_fetch\_helpers.jl](https://github.com/oxinabox/DataDeps.jl/blob/c5256065e4396a49e43c680e35e68357a6eb01f3/src/post_fetch_helpers.jl)
should handle all formats and all operating systems
Which was inturn taken and adapted from some code in BinDeps.jl