# Reading .csv.gz with CSV does not find readavailable(::GZipStream)

**URL:** https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966
**Category:** Data
**Tags:** csv
**Created:** [August 25, 2019, 5:24pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966 "2019-08-25T17:24:46Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![hmmueller](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hmmueller/32/9617_2.png) [@hmmueller](https://discourse.julialang.org/u/hmmueller)
#### Post date: [August 25, 2019, 5:24pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966/1 "2019-08-25T17:24:46Z")

</div>

I had an original piece of code like this, which worked nicely:

```julia
        filepaths = [joinpath(root, f)
                    for (root, dirs, files) in walkdir(root)
                    for f in files[occursin.(fnfeature, files) .& occursin.(r"csv$", files)]]
        df = let OptFloat64=Union{Missing, Float64}, OptInt32=Union{Missing, Int32}
            reduce(vcat, [CSV.read(fp,
                        header=[:domain, :host, :feature, :oid, :largeversion, :clientid,
                                :from, :to, :aggrlevel, :firstocc, :lastocc, :livesuntil,
                                :ct, :sum, :min, :max, :g_lower, :g_upper, :g_ct, :g_sum],
                        types=[String, String, String, Int64, String, String,
                                DateTime, DateTime, Int8, DateTime, DateTime, DateTime,
                                Int32, Float64, Float64, Float64, OptFloat64, OptFloat64, OptInt32, OptFloat64],
                        delim='|')
                        for fp in filepaths]) |> DataFrame
        end

```

For reading csv.gz instead, using kmundnic’s suggestion at [stackoverflow](https://stackoverflow.com/questions/52590792/how-do-i-read-a-gzipped-csv-file-in-julia), I rewrote this (so that I’d not have to learn CSVFiles …) as

```julia
        filepaths = [joinpath(root, f)
                    for (root, dirs, files) in walkdir(root)
                    for f in files[occursin.(fnfeature, files) .& occursin.(r"csv.gz$", files)]]
        df = let OptFloat64=Union{Missing, Float64}, OptInt32=Union{Missing, Int32}
            reduce(vcat, [GZip.open(fp, "r") do io
			      CSV.read(io,
				 header=[:domain, :host, :feature, :oid, :largeversion, :clientid,
				         :from, :to, :aggrlevel, :firstocc, :lastocc, :livesuntil,
					 :ct, :sum, :min, :max, :g_lower, :g_upper, :g_ct, :g_sum],
				 types=[String, String, String, Int64, String, String,
					 DateTime, DateTime, Int8, DateTime, DateTime, DateTime,
					 Int32, Float64, Float64, Float64, OptFloat64, OptFloat64, OptInt32, OptFloat64],
				 delim='|')
			    end
                        for fp in filepaths]) |> DataFrame
        end

```

However, with this I get

```julia
ERROR: LoadError: MethodError: no method matching readavailable(::GZipStream)
Closest candidates are:
  readavailable(::Base.Filesystem.File) at filesystem.jl:199
  readavailable(::IOStream) at iostream.jl:396
  readavailable(::Base.AbstractPipe) at io.jl:243
  ...
Stacktrace:
 [1] write(::Base.GenericIOBuffer{Array{UInt8,1}}, ::GZipStream) at .\io.jl:579

```

☹ - what’s it that I don’t understand? Thanks for help!

// That OptFloat thing is unnecessary, isn’t it? - as the DataValues behind a DataFrame handle “empty values” anyway … But so be it, for the moment …

---

<div class="post-metadata">

### Author: ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)
#### Post date: [August 26, 2019, 4:14pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966/2 "2019-08-26T16:14:33Z")

</div>

The problem is that the Gzip.jl package doesn’t properly implement the IO interface from Base and has received very little maintenance over the last few years. I’d recommend using [https://github.com/bicycle1885/CodecZlib.jl](https://github.com/bicycle1885/CodecZlib.jl) instead, which is actively maintained and includes the proper interfaces for CSV.jl.

---

<div class="post-metadata">

### Author: ![js135005](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/js135005/32/8219_2.png) [@js135005](https://discourse.julialang.org/u/js135005)
#### Post date: [August 26, 2019, 4:18pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966/3 "2019-08-26T16:18:18Z")

</div>

CodecZlib.jl is also much faster than Gzip.jl. I use CodecZlib in production and am very happy with it.

---

<div class="post-metadata">

### Author: ![hmmueller](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hmmueller/32/9617_2.png) [@hmmueller](https://discourse.julialang.org/u/hmmueller)
#### Post date: [August 26, 2019, 7:21pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966/4 "2019-08-26T19:21:08Z")

</div>

Many thanks - I’ll try it tomorrow!

---

<div class="post-metadata">

### Author: ![hmmueller](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hmmueller/32/9617_2.png) [@hmmueller](https://discourse.julialang.org/u/hmmueller)
#### Post date: [August 28, 2019, 5:49pm UTC](https://discourse.julialang.org/t/reading-csv-gz-with-csv-does-not-find-readavailable-gzipstream/27966/5 "2019-08-28T17:49:38Z")

</div>

… and I’m happy now, too. Thanks!
