# Gzipped (.csv.gz) writing?

**URL:** https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767
**Category:** Data
**Created:** [June 18, 2018, 10:06pm UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767 "2018-06-18T22:06:14Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)
#### Post date: [June 18, 2018, 10:06pm UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/1 "2018-06-18T22:06:14Z")

</div>

What is or will be the recommended way to write a data frame to a .csv.gz file?

(I tried `GZip.open` with `writetable` but this failed.)

regards,

/iaw

---

<div class="post-metadata">

### Author: ![js135005](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/js135005/32/8219_2.png) [@js135005](https://discourse.julialang.org/u/js135005)
#### Post date: [June 18, 2018, 10:08pm UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/2 "2018-06-18T22:08:35Z")

</div>

Have you tried Libz or CodecZlib? Both are very much more efficient than GZip. They also use FileIO and DataStreams interfaces IIRC.

---

<div class="post-metadata">

### Author: ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)
#### Post date: [June 19, 2018, 11:41pm UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/3 "2018-06-19T23:41:13Z")

</div>

No, I have not, and I would be happy to try it. Alas, it is not clear how.

Libz tells me that it is deprecated in favor of CodecZlib. CodeZlib has practically no documentation, examples, etc. I can’t even figure out if codeczlib has its own compression format (and/or whether there are unix utility for it, too; gzip has gzcat, gzgrep, and more.)

---

<div class="post-metadata">

### Author: ![js135005](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/js135005/32/8219_2.png) [@js135005](https://discourse.julialang.org/u/js135005)
#### Post date: [June 19, 2018, 11:48pm UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/4 "2018-06-19T23:48:22Z")

</div>

I’ll try to post examples. I’m using both quite successfully (even though Libz says it’s deprecated). Not near my laptop right now.

---

<div class="post-metadata">

### Author: ![janfrancu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/janfrancu/32/5673_2.png) [@janfrancu](https://discourse.julialang.org/u/janfrancu)
#### Post date: [October 9, 2018, 6:56am UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/5 "2018-10-09T06:56:07Z")

</div>

Since @js135005 did not post any example, I would like to add some for future visitors of this thread. The issue with CodecZlib is that it is really interconnected with `TranscodingStreams`, where most of the examples are found, see [https://bicycle1885.github.io/TranscodingStreams.jl/stable/examples.html](https://bicycle1885.github.io/TranscodingStreams.jl/stable/examples.html).  
In order to write DataFrame into a gzipped csv, I use the following lines.

```julia
df = DataFrame(...)
open(GzipCompressorStream, "table.csv.gz", "w") do stream
    CSV.write(stream, df)
end

```

Reading is very similar.

```julia
df = open(GzipDecompressorStream, "table.csv.gz", "r") do stream
    CSV.read(stream)
end

```

---

<div class="post-metadata">

### Author: ![Ajaychat3](https://avatars.discourse-cdn.com/v4/letter/a/ecd19e/32.png) [@Ajaychat3](https://discourse.julialang.org/u/Ajaychat3)
#### Post date: [October 9, 2018, 7:28am UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/6 "2018-10-09T07:28:44Z")

</div>

@janfrancu: Is it possible to define compression level and if so, how. I do see some mention of level in codeczlib(compression.jl) but not able to apply.

---

<div class="post-metadata">

### Author: ![janfrancu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/janfrancu/32/5673_2.png) [@janfrancu](https://discourse.julialang.org/u/janfrancu)
#### Post date: [October 9, 2018, 8:13am UTC](https://discourse.julialang.org/t/gzipped-csv-gz-writing/11767/7 "2018-10-09T08:13:35Z")

</div>

It would be nice to be able to pass the keyword argument to `GzipDecompressorStream` or `GzipCompressorStream` like this

```julia
open(GzipCompressorStream(;level = -1), "table.csv.gz", "w")

```

however AFAIK that is not possible at the moment. This being said, I believe, that you can still construct and open the stream without the closure but with the named arguments.  
I am still figuring this pipeline myself, therefore I cannot really give you the how to guide. I have stumbled across this topic and figured out the issue, so I just wanted to share what I know.
