# GzipDecompressionStream compared to GZip.jl?

**URL:** https://discourse.julialang.org/t/gzipdecompressionstream-compared-to-gzip-jl/5442
**Category:** General Usage
**Created:** [August 18, 2017, 1:45am UTC](https://discourse.julialang.org/t/gzipdecompressionstream-compared-to-gzip-jl/5442 "2017-08-18T01:45:36Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![freeboson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/freeboson/32/2198_2.png) [@freeboson](https://discourse.julialang.org/u/freeboson)
#### Post date: [August 18, 2017, 1:45am UTC](https://discourse.julialang.org/t/gzipdecompressionstream-compared-to-gzip-jl/5442/1 "2017-08-18T01:45:36Z")

</div>

How does the `GzipDecompressionStream` compare to `GZip.jl` for line by line reading? Also, thank you for making this very useful package!

---

<div class="post-metadata">

### Author: ![bicycle1885](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bicycle1885/32/107_2.png) [@bicycle1885](https://discourse.julialang.org/u/bicycle1885)
#### Post date: [August 18, 2017, 4:26am UTC](https://discourse.julialang.org/t/gzipdecompressionstream-compared-to-gzip-jl/5442/2 "2017-08-18T04:26:22Z")

</div>

Both packages support line-based IO operations but there are some differences.

`GzipDecompressionStream` can wrap any I/O stream while `GZipStream` can only read  
data from a file. For example, if your gzip data come from other process  
via pipe, you can use `GzipDecompressionStream` but cannot use `GZipStream`.

```
julia> using CodecZlib

julia> pipe, proc = open(`cat movies.csv.gz`)
(Pipe(RawFD(-1) closed => RawFD(17) open, 0 bytes waiting), Process(`cat movies.csv.gz`, ProcessRunning))

julia> stream = GzipDecompressionStream(pipe)
TranscodingStreams.TranscodingStream{CodecZlib.GzipDecompression,Pipe}(<state=idle>)

julia> readline(stream)
"\"\",\"title\",\"year\",\"length\",\"budget\",\"rating\",\"votes\",\"r1\",\"r2\",\"r3\",\"r4\",\"r5\",\"r6\",\"r7\",\"r8\",\"r9\",\"r10\",\"mpaa\",\"Action\",\"Animation\",\"Comedy\",\"Drama\",\"Documentary\",\"Romance\",\"Short\""

julia> readline(stream)
"\"1\",\"\$\",1971,121,NA,6.4,348,4.5,4.5,4.5,4.5,14.5,24.5,24.5,14.5,4.5,4.5,\"\",0,0,1,1,0,0,0"

julia> close(stream)

julia> pipe
Pipe(RawFD(-1) closed => RawFD(-1) closed, 278528 bytes waiting)

julia> proc
Process(`cat movies.csv.gz`, ProcessSignaled(13))

```

In terms of performance, on my local benchmark, CodecZlib.jl is significantly  
faster than GZip.jl:

```
shell> ls -lh
total 4896
-rw-r--r-- 1 kenta staff 52B Aug 7 2016 1000x2.csv.gz
-rw-r--r-- 1 kenta staff 971K Aug 7 2016 movies.csv.bz2
-rw-r--r-- 1 kenta staff 1.4M Aug 7 2016 movies.csv.gz

shell> gzip -cd movies.csv.gz | wc -l
   58789

julia> using BenchmarkTools

julia> using GZip

julia> @benchmark foreach(identity, eachline(GZip.gzopen("movies.csv.gz")))
BenchmarkTools.Trial:
  memory estimate: 37.84 MiB
  allocs estimate: 530031
  --------------
  minimum time: 279.652 ms (1.81% GC)
  median time: 287.000 ms (1.68% GC)
  mean time: 287.984 ms (1.50% GC)
  maximum time: 305.171 ms (0.91% GC)
  --------------
  samples: 18
  evals/sample: 1

julia> using CodecZlib

julia> @benchmark foreach(identity, eachline(GzipDecompressionStream(open("movies.csv.gz"))))
BenchmarkTools.Trial:
  memory estimate: 24.55 MiB
  allocs estimate: 236274
  --------------
  minimum time: 38.299 ms (3.82% GC)
  median time: 39.422 ms (4.37% GC)
  mean time: 40.007 ms (4.87% GC)
  maximum time: 47.506 ms (4.79% GC)
  --------------
  samples: 125
  evals/sample: 1

```

---

<div class="post-metadata">

### Author: ![freeboson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/freeboson/32/2198_2.png) [@freeboson](https://discourse.julialang.org/u/freeboson)
#### Post date: [August 22, 2017, 2:00am UTC](https://discourse.julialang.org/t/gzipdecompressionstream-compared-to-gzip-jl/5442/3 "2017-08-22T02:00:34Z")

</div>

Yeah, this is absolutely great. I am happily using `XzDecompressionStream` in my project, and it’s working very well.
