# How to concatenate multiple \`.gz\` files within Julia?

**URL:** https://discourse.julialang.org/t/how-to-concatenate-multiple-gz-files-within-julia/44975
**Category:** General Usage
**Tags:** question
**Created:** [August 15, 2020, 4:08am UTC](https://discourse.julialang.org/t/how-to-concatenate-multiple-gz-files-within-julia/44975 "2020-08-15T04:08:48Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![biona001](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/biona001/32/16497_2.png) [@biona001](https://discourse.julialang.org/u/biona001)
#### Post date: [August 15, 2020, 4:08am UTC](https://discourse.julialang.org/t/how-to-concatenate-multiple-gz-files-within-julia/44975/1 "2020-08-15T04:08:48Z")

</div>

I want to combine multiple `.gz` files into 1 within Julia. The command to do so in the command line is:

```julia
cat file1.gz file2.gz file3.gz > allfiles.gz

```

But the following does not work:

```julia
files = ["file1.gz", "file2.gz", "file3.gz"]
run(pipeline(`cat $files`, stdout="allfiles.gz"))

```

The function doesn’t throw any error but `allfiles.gz` is corrupt. Is this a bug?

---

<div class="post-metadata">

### Author: ![biona001](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/biona001/32/16497_2.png) [@biona001](https://discourse.julialang.org/u/biona001)
#### Post date: [August 15, 2020, 4:30am UTC](https://discourse.julialang.org/t/how-to-concatenate-multiple-gz-files-within-julia/44975/2 "2020-08-15T04:30:29Z")

</div>

I figured it out as I was trying make a MWE. The following does not work:

```julia
using CodecZlib
for i in 1:3
    file = "file$i.gz"
    x = rand(2, 2)
    io = GzipCompressorStream(open(file, "w"))
    write(io, "hello, testing from i = $i")
end
files = ["file1.gz", "file2.gz", "file3.gz"]

# runs without error but file corrupted
run(pipeline(`cat $files`, stdout="allfiles.gz")) 

```

But this works:

```julia
using CodecZlib
for i in 1:3
    file = "file$i.gz"
    x = rand(2, 2)
    io = GzipCompressorStream(open(file, "w"))
    write(io, "hello, testing from i = $i")
    close(io) #need to close stream
end
files = ["file1.gz", "file2.gz", "file3.gz"]
run(pipeline(`cat $files`, stdout="allfiles.gz")) # works

```

The difference is that in the first case I forgot the close the stream. Sorry!

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 15, 2020, 8:18am UTC](https://discourse.julialang.org/t/how-to-concatenate-multiple-gz-files-within-julia/44975/3 "2020-08-15T08:18:26Z")

</div>

Note that your problem is simply concatenating files, and has nothing specific to gz (you are simply concatenating compressed files _without_ compressing/decompressing, which is fine, you can do it with gz).

So you would simply read and write them, eg

```julia
open("allfiles.gz", write = true) do io # UNTESTED, just a sketch
    for file in ["file1.gz", "file2.gz", "file3.gz"]
        write(io, read(file, String))
    end
end

```

or do some variant of block based IO when the files are larger.
