HTTP.jl v2.12 incorrect header check (code: -3)

General Usage
,
1

I’m trying to migrate from v1.11 to v2.12 to call an Inmation rest API. It worked for v1.11 but not v2.12, and the error stacktrace I received is shown below.

ERROR: CodecZlib.ZlibError("incorrect header check (code: -3)")
Stacktrace:
  [1] changemode!(stream::TranscodingStreams.TranscodingStream{CodecZlib.ZlibDecompressor, HTTP._BodyIO{…}}, newmode::Symbol)
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:794
  [2] callprocess(stream::TranscodingStreams.TranscodingStream{…}, inbuf::TranscodingStreams.Buffer, outbuf::TranscodingStreams.Buffer)
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:707
  [3] fillbuffer(stream::TranscodingStreams.TranscodingStream{CodecZlib.ZlibDecompressor, HTTP._BodyIO{…}}; eager::Bool)
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:624
  [4] fillbuffer
    @ C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:610 [inlined]
  [5] sloweof(stream::TranscodingStreams.TranscodingStream{CodecZlib.ZlibDecompressor, HTTP._BodyIO{HTTP.H2Body}})
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:222
  [6] eof
    @ C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:213 [inlined]
  [7] readbytes!(stream::TranscodingStreams.TranscodingStream{…}, b::Vector{…}, nb::Int64)
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:401
  [8] _read_all_response_bytes(io::TranscodingStreams.TranscodingStream{…}, limit::Int64)
    @ HTTP C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:1391
  [9] #_consume_incoming_response!##0
    @ C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:1817 [inlined]
 [10] _with_response_reader(f::HTTP.var"#_consume_incoming_response!##0#_consume_incoming_response!##1"{…}, incoming::HTTP._IncomingResponse{…}, decompress::Nothing)
    @ HTTP C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:1755
 [11] _consume_incoming_response!(incoming::HTTP._IncomingResponse{…}, sink::Nothing, decompress::Nothing, max_decompressed_size::Int64)
    @ HTTP C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:1815
 [12] request(method::String, url::String, h::Vector{…}, b::String; headers::Vector{…}, body::String, basicauth::Nothing, retry::Bool, retries::Int64, retry_non_idempotent::Bool, retry_if::Nothing, respect_retry_after::Bool, retry_bucket::Bool, status_exception::Bool, redirect::Bool, redirect_limit::Nothing, redirect_method::Nothing, forwardheaders::Bool, proxy::HTTP._UseTransportProxy, cookies::Bool, cookiejar::Nothing, query::Nothing, response_stream::Nothing, decompress::Nothing, max_decompressed_size::Int64, sse_callback::Nothing, max_sse_line_bytes::Int64, max_sse_event_bytes::Int64, client::Nothing, context::Nothing, connect_timeout::Nothing, request_timeout::Int64, response_header_timeout::Int64, read_idle_timeout::Int64, write_idle_timeout::Int64, expect_continue_timeout::Nothing, readtimeout::Nothing, copyheaders::Bool, pool::Nothing, canonicalize_headers::Nothing, detect_content_type::Nothing, observelayers::Nothing, retry_delays::Nothing, retry_check::Nothing, sslconfig::Nothing, socket_type_tls::Nothing, logerrors::Nothing, logtag::Nothing, trace::Nothing, verbose::Nothing, require_ssl_verification::Bool, protocol::Symbol)
    @ HTTP C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:2302
 [13] request
    @ C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:2130 [inlined]
 [14] #post#155
    @ C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:2551 [inlined]
 [15] post(::String, ::Vector{Pair{String, String}}, ::String)
    @ HTTP C:\Users\usr\.julia\packages\HTTP\Iguyh\src\http_client.jl:2549

I used fairly generic headers:

headers = [
    "Content-Type" => "application/json",
    "Accept" => "application/json",
    "username" => credentials.username,
    "password" => credentials.password,
]

Is there something I’m doing wrong here, or is it just that there are incompatibilities between HTTP.jl v2.x and the HTTP/1.1 protocol that Inmation uses? I can’t make heads or tails of this error message or why it’s appearing.

2

Thanks for the report — this is an HTTP.jl bug. HTTP.jl 2.x advertises Accept-Encoding: gzip, deflate (1.x only asked for gzip), and your server answers with “deflate” data in the raw form .NET’s DeflateStream produces, which 2.x’s decoder didn’t accept. Fixed in Decode raw DEFLATE bodies sent as Content-Encoding: deflate - Pull Request #1511 - JuliaWeb/HTTP.jl - GitHub. Until it’s released, adding "Accept-Encoding" => "gzip" to your headers should get you back to the 1.11 behavior.

3

Thanks for getting back to me on this. I’m now using

headers = [
    "Accept-Encoding" => "gzip",
    "Content-Type" => "application/json",
    "Accept" => "application/json",
    "username" => credentials.username,
    "password" => credentials.password,
]

But I’m getting a slightly different error now with the exact same stack trace

ERROR: ZlibError: incorrect header check (code: -3)
Stacktrace:
  [1] changemode!(stream::TranscodingStreams.TranscodingStream{CodecZlib.ZlibDecompressor, HTTP._BodyIO{…}}, newmode::Symbol)
    @ TranscodingStreams C:\Users\usr\.julia\packages\TranscodingStreams\O3BYF\src\stream.jl:794
4

Sorry, that workaround assumed your server honors Accept-Encoding, and it doesn’t: it answers with deflate even when only gzip is requested. Inmation’s own docs decode these responses as raw DEFLATE (their Lua client example uses LibDeflate:DecompressDeflate), which is the form HTTP.jl 2.12 can’t decode yet; Decode raw DEFLATE bodies sent as Content-Encoding: deflate - Pull Request #1511 - JuliaWeb/HTTP.jl - GitHub fixes that. Until it’s released, this works on 2.12:

using CodecZlib  # ] add CodecZlib
r = HTTP.post(url, headers, body; decompress = false)
data = HTTP.header(r, "Content-Encoding") == "deflate" ? transcode(DeflateDecompressor, r.body) : r.body

If you have a minute, could you also try your original call with protocol = :h1? HTTP.jl 1.x only spoke HTTP/1.1, and I suspect your server only compresses over HTTP/2, which would explain why 1.11 worked.

5

Yup, that did the trick. Setting protocol =: h1 worked. At the moment, compression is a nice to have, but not necessary. I’ve been living without it for years but I’ll keep an eye out for this! Thanks a lot!

6

I just updated and it works without protocol =: h1. I guess I get compression now. Thanks even more!