[ANN] HTTP.jl 2.0 release and new package Reseau.jl

Package Announcements
1

Hello! I’m excited to announce two related networking package updates: Reseau.jl and HTTP.jl 2.0.

The short version: HTTP.jl 2.0 is a pretty substantial internals rewrite, now built on top of Reseau.jl: a new lower-level networking package for libuv-free TCP/TLS IO primitives. On top of that, HTTP.jl now has rewritten pure-Julia HTTP/1.1 and HTTP/2 client/server support.

Most of the high-level interface for HTTP remains the same: HTTP.request, HTTP.get, HTTP.post, HTTP.serve, routing, streaming, SSE, and WebSocket flows should remain largely the same. Some deprecated/old keyword arguments have been removed or updated in new/better ways. HTTP.download was also removed in favor of Downloads.download. Because of the amount of internal churn, I felt a 2.0 release was appropriate, but I’m also very open to “unbreaking” anything that may have broken unintentionally. If you run into issues, or find something that seems like it should work, open an issue and we can probably add compat/fix.

A few notable pieces:

  • HTTP/1.1 client/server support was rewritten on top of the new Reseau-based transport layer.
  • HTTP/2 client/server support is now first-class.
  • Reseau provides the new network IO foundation: native TCP/TLS, a Go-inspired IO readiness poller, and a clean wait/notify boundary at the lowest levels that doesn’t require a global IO lock.
  • HTTP.jl 2.0 now requires Julia 1.10+.
  • Both Reseau.jl and HTTP.jl now maintain broad trim-compiled workloads as part of their test suites (though see some discussion/forth-coming Julia fixes for true HTTP trim-compiled workloads to fully work).

One personal goal I’ve had for a while was getting HTTP.jl to the point where it could actually saturate large cloud network links. HTTP.jl 1.x, because of libuv’s global IO lock around network operations, would top out around 10-12 Gbps no matter how much I tuned multithreading. With the Reseau-backed stack, I was able to hit around 46 Gbps from a single Julia process on a machine advertised with 40 Gbps network capacity, using concurrent cloud-store downloads. That’s probably not going to matter for most day-to-day HTTP.jl users, but it removes a ceiling that had been bothering me for a long time and opens up some more serious networking use cases.

Another major goal was making the stack friendlier to trim compilation / juliac. I spent a while pursuing an AWS CRT-based version of this. It got pretty far: cross-platform, very fast, and able to hit the same kind of throughput numbers. But the architecture ended up being a rough fit for trim compilation. CRT wants per-thread event loops where IO work and callbacks run on network threads, which meant a lot of Julia callback/function shuffling across thread boundaries. I kept finding myself needing hack-upon-hack to make that world compile cleanly.

The approach Reseau ended up taking is closer to Go’s network poller model. Julia threads can optimistically do their own IO; when they need to wait, they register the fd with a single readiness poller thread, park, and get woken back up when the IO event is ready. That ended up being much simpler, easier to reason about, and much easier to make trim-compileable while still hitting the performance goals.

There is still one known Julia Task wrinkle: trim-compiled background Tasks currently need a Julia fix. There’s an open proposal/patch for that, and with that patched Julia I can run fully trim-compiled HTTP client/server workloads. The important change in this release is that trim-compileability is now something the packages test and preserve and will maintain going forward.

For breaking changes, I’m hoping the actual surface area is fairly manageable. Since this is a major version, downstream packages need to opt in via compat, so it should not silently break existing environments. Some of the more visible changes:

  • HTTP.download is gone; use Downloads.download / Base.download, or HTTP.request / HTTP.open when you need HTTP-specific control.
  • RequestContext is now typed request state rather than just a plain dictionary, though migration helpers exist.
  • Client retry/timeout/pooling configuration has been cleaned up around the new Client / Transport APIs.
  • Old undocumented layer/parser/connection internals have been completely rewritten, so migrating will be case-by-case.

There is a migration guide in the HTTP.jl docs with more detail.

I’ve had a few personal projects running on this branch for a while without hiccups, and I’ll be working through popular downstream packages to help with compat updates where needed. I’d love to hear about any issues people run into, especially from package maintainers, folks using HTTP.jl in servers, and anyone experimenting with juliac / trim compilation.

Huge thanks to everyone who has tested, reviewed, asked good questions, or filed issues along the way. This was a lot of work over a long period, and it feels great to keep pushing things forward.

2

Super cool! Have you tested anything related to io_uring? It seems that the performance with epoll is close to optimal so not sure if it’s even worth to explore at this point.

3

Wonderful! From here how big of a lift would QUIC support be?

4

No, haven’t done any extensive experimentation or testing w/ io_uring. Tried to stay very close to the go netpoll shape of things.

I think it could be worth experimenting with, as we already do IOCP on the windows side, so it would be conceptually similar; it’d be interesting to see if certain workloads would benefit from it vs. epoll.

@Oscar_Smith, yeah it’s a good question. I didn’t even really do full UDP support in Reseau yet as I was mostly focused on TCP → TLS → HTTP. UDP itself wouldn’t be too hard to add/layer in, and we already have a bunch of native TLS, so that helps w/ the QUIC transport side of things.

I think the trickiest bit is just getting a solid testing framework/reference implementation to go off of. That was really the big enabler for netpoll/TCP/TLS/HTTP/1/2 was having all this really solid go code (and tests!) to base everything off. It helps enormously in the overall confidence of the implementation.

5

This is huge! Among the many other improvements you mentioned, my favorite bit is that this drops the dependency on MBedTLS v2 — a very old, unsupported, and known to be vulnerable version.

Thanks for your hard work here @quinnj!

6

Having native HTTP/2 in Julia makes many things much easier, including finishing gRPCServer.jl. The Julia package ecosystem just got a massive boost. Really appreciate all of the work that went into shipping this!

7

@quinnj I don’t suppose you’ve seen io_uring, kTLS and Rust for zero syscall HTTPS server?

8

Hi,

Great to see the progress. How confident are you in Reseau.jl? It’s quite a lot of code, most of it probably generated by AI. Can you briefly describe the effort going into the verification and validation side?

Thanks for the package.

9

Valid concern/question: I’m realizing it’s a bit buried at the bottom of the README.md, but I tried to outline the package development approach for Reseau here. To repeat here, yes, there was heavy use of AI in the development of Reseau, and my confidence/justification comes from:

  • Ability to base and port the code from very well-established Go standard library code, largely viewed as one of the best vertically integrated network stacks; especially including test suites
  • Always hand/manually reviewing all code
  • It wasn’t completely explicit in my original announcement post above, but the code in its current state is actually major iteration #7 or so. Having developed in Julia for over a decade, you get a feel for what works, what doesn’t, and if all the goals are being met. I had quite a few iterations that worked in one way, but ended up not in other ways. I’ve appreciated how AI can enable faster iteration on architectures where this kind of project would have taken years and instead took more like 6-9 months. It helps take care of the lower-level details while I was able to riff on different designs until it all settled in a way that felt “right”.
  • Similar to the Go argument, this is also just a space that has a lot of reference code, official protocol specifications, explicit guidelines/rules, etc. which also makes it a space more amenable to AI development vs. other areas. There’s more verification/reference options than when I’m developing a data-munging framework or writing something else more unstructured or creatively demanding.

I won’t get too long-winded here on AI pontificating, but I really just see it as a huge boost and helpful tool. I’m still the engineer, responsible for the code, and it’s my name on commits/repo. AI helps, but we’re still people sharing software.

10

I realize that most questions regarding the use of AI often come with a critical undertone - this was not my intended tone! I’m in the same boat to see AI as a way to materialize different approaches, and so disarming the sunken cost fallacy, allowing for the best solution to be chosen. The “Always hand/manually reviewing all code” sentence is what I had hoped for.

11

Nice! What does the name Reseau mean?

Am I right in thinking Reseau provides a native implementation of TCP, except with TLS provided by OpenSSL?

Any plans for HTTP/3, which is UDP/QUIC based?

12

As far as I understand, Reseau.jl is the underlying networking layer. And it means “network” in French, so the name sounds in tune :slight_smile:

13

Very exciting, congratulations! I’m really happy to see so much effort going into stability, porting the Go stdlib implementation and tests sounds like a smart idea!

14

Nice! Some comments:

  1. The kind of code implemented in Reseau is scary! This is very explicitly security code and the other side of the ethernet cable is enemy country. Do you really need to do e.g. ans1 yourself (as opposed to asking openssl to deal with the finicky parts)?
  2. Regarding confidence in your code: Good that you personally reviewed all the code that AI helped write/port! But that’s the minimum. Confidence in such code also needs a lot of (security-) testing and review. Have you looked at things like GitHub - tls-attacker/TLS-Anvil: TLS-Anvil, a fully automated TLS testsuite for client and servers. · GitHub and similar? Fuzzing? Established libraries in that space spend a lot of effort on process for this kind of testing, far more than on writing the code.
  3. Minor nitpicks / code-review on the ASN.1 parsing (that’s what I looked at for ~10 minutes):
    1. DER “distinguished” encoding (as opposed to BER) requires that the leading length byte is nonzero Reseau.jl/src/tls/x509.jl at 2ff4511583d2eaca2fb4bed95c6b919232693b15 · JuliaServices/Reseau.jl · GitHub – otherwise you could have used one less length-bytes. Afaiu most TLS implementations reject such invalid certificates. I’d at least leave a source-code comment explaining the decision to accept this.
    2. I like the defensive way the asn1 parsing is written (I’d have preferred if it wasn’t written at all). A good opportunity you missed is to use an internal API like _asn1_expect_tlv(bytes::AbstractVector{UInt8}, pos::Int, expected_tag::UInt8, upper_bound::Int)::NTuple{3, Int} that always takes an upper bound until where the value can be read. That is to mitigate a typical issue when reading Tag-length-value nested structs: Catch malformed structs where (tag:?, length: 10, value:(tag:?, length: 20, value)), ..., i.e. where the inner nested structure escapes the containing outer structure and takes bytes from the next one. I think you have checks that eventually deal with these guys, but best practice would be to put that into your lowest-level API. Otherwise reviewing is much harder (are any invalid structures accepted? Are these a security-problem? Can situations happen where attacker-controlled and secret data is adjacent and timing / error messages leak confidential data?).

If you have not yet run a bunch of automated test-suites for common tls / x509 pitfalls, then I think it is worthwhile to write down your expectations beforehand, in order to better calibrate your confidence in your code / development process. That way you gain far more insight than just “oh, bug X, bug Y, bug Z”.

PS (one day layer) for those reading along this thread and not github:

@quinnj + claude already implemented my “minor nitpicks”, and did some research on available testing suites. Wow is that fast, :+1:!

Especially wycheproof and BoGo are very good, I should have recommended them instead of tls-anvil, they just slipped my mind.

…I would have kept the minor nitpicks unfixed for now as a “holdout set”, in order to calibrate confidence in the upcoming conformance tests, and only addressed them afterwards. A non-comprehensive drive-by review from discourse can calibrate confidence in your code and help to highlight weaknesses in development process / API-design / testing, but cannot meaningfully decrease the density of bugs. It’s sometimes nice to keep some non-critical known bugs stashed for “testing the test”.

15

I’m probably very mistaken on something, but how does a libuv-free Reseau work with Julia’s libuv-based task scheduling e.g. a task waiting for a request being replaced by a ready task on an OS thread?

16

Good question. I can be a bit more precise in the wording here: by “libuv-free” I mean Reseau does not use libuv as the socket polling / TCP transport layer, not that it bypasses Julia’s task scheduler entirely (I did take a look around on the task scheduling side of things, but in reality, IO perf has never been limited/impacted very much by task scheduling compare w/ the global IO lock).

Reseau sockets are not Base.TCPSocket/libuv handles. Reseau creates nonblocking OS sockets, registers them with its own IOPoll, and waits on the native backend directly: epoll + eventfd on Linux, kqueue on macOS/BSD, and IOCP on Windows. When a Julia task needs to wait for socket readiness, Reseau records the current task in a small waiter object and parks it with the low-level wait()/schedule(task) protocol.

When the native poller (single Reseau-managed foreign thread) sees readiness, it wakes the waiter by calling schedule(task), so Julia’s scheduler is still what decides when and where that task runs next. The distinction is that libuv is not the component watching the socket fd or driving the TCP/TLS readiness path; Reseau does that itself and then hands control back to Julia by scheduling the parked task.

This model of “every task does its own optimisitic IO; park on the poller when necessary” ends up being very lightweight in terms of the centralized coordination required and scales really well; in my largest experiment we had dozens of threads on a huge machine all coordinating smoothly, maxing out the 40Gbps network bandwidth.

Anyway, little bit long-winded there, but hopefully it helps clarify.

17

Oops, forgot to properly reply here: yes, native-per-platform TCP implementations. TLS is also mostly pure Julia, but utilizes OpenSSL_jll for the lowest level crypto hashing/algorithms.

Scroll up for my previous answer here.

18

Exactement :smile: