How to read a gzipped HTTP response?

Hi,

I’m working with a third-party API and apparently Accept-Encoding: identity is not possible, so I am receiving a gzipped response body whether I like it or not.

I’ve been looking at TranscodingStreams.jl for a while now and still not any closer to figuring this out so could use a nudge in the right direction. Can someone help me decode a gzipped HTTP response body? :pray:

PS: I’ve also seen Gzip.jl, but a little hesitant to use it since it doesn’t seem to be maintained. Please correct me if I’m wrong. Maybe it is solid and doesn’t need maintenance? :sweat_smile:

As usual, the answer is right there is the docs :slight_smile:

https://bicycle1885.github.io/TranscodingStreams.jl/stable/examples/#Transcode-data-in-one-shot-1

decompressed = transcode(GzipDecompressor, r.body)
4 Likes

GZip.jl is a low-level wrapper around the gzip functionality in libz. Because of this, there’s not much to maintain–it should generally work fine, but won’t do anything fancy and might not give you the fastest results (you would have to test different buffer sizes).

TranscodingStreams.jl (and the underlying CodecZlib.jl) are a bit more modern and flexible, and should be faster (by default, at least).

Cheers,
Kevin

1 Like

I use TranscodingStreams and CodecZlib in production. I have found it to be much faster than GZip.

2 Likes

Just a note: CodecZlib.jl is still far from optimal because it uses binaries compiled by BinaryBuilder.jl. I found that those are much slower (~2-3 times) than those of systems or compiled on the target architecture. I believe Elliot will resolve or mitigate the problem in the near future.

1 Like