# HTTP.jl doesn't seem to be good at handling over 1k concurrent requests, in comparison to an alternative in Python?

**URL:** <https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281>\
**Category:** Web Stack\
**Created:** [April 27, 2020, 3:53am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281 "2020-04-27T03:53:38Z")\
**Posts on this page:** 11\
**Page:** 2

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [August 13, 2020, 3:15pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/21 "2020-08-13T15:15:05Z")

</div>

Is some time since discussion, but curious what thoughts are?

Basically most of the popular “_fast_” web frameworks use `http-parser` (a C library).  
List includes Bjoern, Vibora or NodeJS, among others.

So there are 2 solutions.

1 - Use `ccall` to use `http-parser` for request handling. This is not ideal, as it will beat fact that Julia is supposed to be equivalently performant to C. Still, writing back to incoming connection functionality will need to be implemented (can rely on `HTTP.jl`).  
2 - Implement logic freshly in Julia. Having done some research, have seen that [`PicoHTTPParser`](https://github.com/h2o/picohttpparser/) is faster than `http-parser`. Again, writing back to incoming connection functionality will need to be implemented (can rely on `HTTP.jl`).

Above mentioned, would recommend 2nd option.  
This would really help to promote Julia’s attractiveness!

What do you think?  
I have started translating `PicoHTTPParser` to Julia

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 13, 2020, 3:39pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/22 "2020-08-13T15:39:47Z")

</div>

> [@yoh-meyers](#):
>
> Use `ccall` to use `http-parser` for request handling.

FYI: There’s an (outdated) Julia wrapper for the/a C [http-parser](https://github.com/joyent/http-parser) library:

> **[GitHub - JuliaWeb/HttpParser.jl: Deprecated! Julia wrapper for...](https://github.com/JuliaWeb/HttpParser.jl)**
>
> Deprecated! Julia wrapper for joyent/http-parser. Contribute to JuliaWeb/HttpParser.jl development by creating an account on GitHub.

It predates the JLL/artifact system (and Julia 1.0), which I think changes things. I see no good reason to avoid good (debugged) C code.

It’s good to have options if you want to port (or wrap) `PicoHTTPParser`, but I think we should concentrate on the future HTTP/3 (and HTTP/2). There are good C libraries for, and this is a big change, and unfeasible to port I think, as the libraries are big. I suggested at some point adding to HTTP.jl but issue was closed as “HTTP.jl is for pure-Julia code only” (I’m not sure too wise, given JLL possibility, at least a competing package could do it):

> <https://github.com/JuliaWeb/HTTP.jl/issues/448>
>
> FYI: Litespeed open sourced code in 2017, and again days ago, 
> 
> I thought I sh…ould let you know, found HPACK.jl which pointed here (and while it claims Julia-only; potentially good-to-have, not unless actually as fast as possible and maintained, and I think we should use be/use this fastest code in "the world").
> 
> They probably have more features, and at least for the case in b) 2 to 12 times faster, so seems Julia should use it:
> 
> a)
> https://blog.litespeedtech.com/2019/09/11/litespeed-quic-http3-open-source/
> 
> "LiteSpeed Technologies is releasing its QUIC and HTTP/3 library under an open-source license. This library, written in the C programming language, contains both client and server logic as well as \[..\]
> The following QUIC and HTTP/3 versions are currently supported:
> \* Q039
> \* Q043
> \* Q046
> \* h3-22
> 
> Q039, Q043, and Q046 are versions of the Google QUIC protocol. Q046 is used by the current version of Chrome (Chrome 76). h3-22 is the experimental precursor to HTTP/3, the next-generation HTTP protocol. We will update the library as new gQUIC and HTTP/3 versions are released."
> 
> b)
> Under MIT licence already ported to Windows and with bug fixes seems to hard to pass up:
> 
> https://github.com/litespeedtech/ls-qpack
> "Fix fast Huffman decoder: detect bad EOS in all cases 10 days ago"
> 
> https://blog.litespeedtech.com/2019/09/11/litespeed-quic-http3-open-source/
> 
> "Our web server outperforms nginx by a factor of 2 to 12 in several readily reproducible benchmarks.
> 
> How do we do it? The truth is, when we are not adding new features, we are optimizing existing code. \[..\]
> Why do we do it? HTTP/2 is on its way to become the dominant web protocol. HTTP/2 performance is crucial for any modern web server. And, of course, we want to be faster than everyone else.
> 
> This article describes one of the optimizations that made ours the fastest HTTP/2 implementation in the world.
> 
> Well-led software projects are not afraid to borrow good ideas, for the hubris of the not-invented-here syndrome often leads to suboptimal implementations. Such is the story of the HPACK Huffman decoder in today’s major web servers. \[..\]
> by Renato Pajarola in his paper Fast Prefix Code Processing. It is significantly faster than prior methods. The idea is to process compressed data several bits — instead of one bit — at a time. Tatsuhiro chose 4 bits as input length. 4 is a good number. \[..\]
> Presented with normal input, the LiteSpeed Fast Huffman Decoder almost doubles the speed of the nginx Huffman decoder. \[..\]
> We also use the Fast Huffman Decoder in our QPACK library, speeding up our HTTP/3 stack. We intend to maintain performance leadership with next-generation HTTP version as well.
> 
> While writing this article, I found two more optimizations that would make the Fast Huffman Decoder even faster. Stay tuned!"
> 
> Older: https://blog.litespeedtech.com/2017/10/02/developers-corner-why-use-litespeed-quic-library/
> 
> See also: https://blog.litespeedtech.com/2019/08/15/litespeed-addresses-http-2-dos-advisories/

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [August 13, 2020, 3:52pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/23 "2020-08-13T15:52:21Z")

</div>

Hello Páll!

I agree with you on focusing on the future (HTTP3).  
Would be more rewarding for the effort.

Only reason I suggested translating `PicoHTTPParser`, is to have less dependencies.  
See that [http-parser](https://github.com/joyent/http-parser) is indeed a translation, only to be depreciated.

Pity to see that `HTTP.jl` is Julia-only.  
It is a good library, but I have in mind a “server-only” lightweight and fast alternative (`HTTP.jl` includes making requests etc, which is not needed for most simple API server applications).

Really have in mind something fast, so that running an API server in Julia, starts to attract non-Julia web coders.  
So if you are open for collaboration, would be happy to start coding an alternative web server library with you.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 14, 2020, 5:27am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/24 "2020-08-14T05:27:01Z")

</div>

Julia certainly seems capable of being fast, so I don’t see why HTTP.jl should be slower than other languages. I do realize the stop the world GC could be an issue if latency needs to be strict.

Has anyone tried profiling HTTP.jl? Where was the time being spent?

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [August 14, 2020, 6:44am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/25 "2020-08-14T06:44:50Z")

</div>

> [@dlakelan](#):
>
> Julia certainly seems capable of being fast, so I don’t see why HTTP.jl should be slower than other languages.

In theory yes. But parsers in other languages use optimisations, such as:

- predicting content of buffer
- better use of underlying hardware

Julia is capable of both of these mentioned points.  
But having quickly parsed the `HTTP.jl` code yesterday, seems this is not case, though buffer is parsed in chunks (and not char by char).

Also I don’t find `HTTP.jl` code base well structured, at first glance.

So to answer your point, yes, though a substantial refactoring would be necessary

---

<div class="post-metadata">

**Author:** ![pixel27](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pixel27/32/8902_2.png) [@pixel27](https://discourse.julialang.org/u/pixel27)\
**Post date:** [August 15, 2020, 7:32pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/26 "2020-08-15T19:32:12Z")

</div>

I’m kind of disappointed with the last posts. I don’t want to come off hostile, but Julia is a new language and we are lucky someone wrote an HTTP server, but you are complaining because the single/first HTTP server that was written natively for Julia isn’t the fastest/best around. I feel like your statements mean that if we can’t build the fastest/best implementation around then why even bother.

Do you feel that the first HTTP server written for C was the fastest/best implementation around? Why do you feel that the first implementation written in Julia should compete with the best implementations C has to offer?

Truthfully when I develop new code I try to be conscious of of performance but my goal is really to make the code bug free with good abstractions and structure. Then if the performance isn’t doing it, I profile and optimize the bottlenecks. As you apply optimizations the code often becomes harder to read and more bug prone. So having the first implementation that is well structured and readable to debug makes life easier.

If the current implementation of HTTP doesn’t meet your needs then please, please, please start a new project to write a better one, or contribute to the current one to improve it’s speed.

---

<div class="post-metadata">

**Author:** ![anon37204545](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon37204545](https://discourse.julialang.org/u/anon37204545)\
**Post date:** [August 15, 2020, 8:22pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/27 "2020-08-15T20:22:46Z")

</div>

> [@pixel27](#):
>
> I’m kind of disappointed with the last posts. I don’t want to come off hostile, but Julia is a new language and we are lucky someone wrote an HTTP server, but you are complaining because the single/first HTTP server that was written natively for Julia isn’t the fastest/best around. I feel like your statements mean that if we can’t build the fastest/best implementation around then why even bother.

I don’t think they are necessarily criticizing anything, just noting what should be made better, for anyone interested.

I’d vouch for writing the interface completely in Julia (as HTTP.jl is) instead of C. Julia code certainly can be speeded up, while C code hardly can be more extensible and accessable.

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [August 16, 2020, 9:07am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/28 "2020-08-16T09:07:53Z")

</div>

> [@anon37204545](#):
>
> I don’t think they are necessarily criticizing anything, just noting what should be made better, for anyone interested.

At least from my side, this was intention.  
Am openly stating what needs to be improved for Julia to be able to serve heavy Devops applications.

> [@yoh-meyers](#):
>
> So if you are open for collaboration, would be happy to start coding an alternative web server library with you.

Indeed I even suggested coding an alternative library.  
I believe that having a healthy discussion can lead to new ideas and collaborations (hence my posts).  
FYI I already started coding a Flask (Python) like [web library](https://github.com/DanceJL/Dance.jl). And if you look at Github tasks you will see that a faster HTTP parser is part of list.

So in conclusion I am in process of coding an alternative server library in Julia, inspired from `PicoHTTPParser` .  
While coding everything in Julia is optimal, I don’t deny that for the HTTP 2/3 stuff, I might simply import C libraries.

Any feedback/comments are welcome

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [October 9, 2020, 5:24am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/29 "2020-10-09T05:24:25Z")

</div>

Hi @aviatesk,

Did you ever do more detailed profiling and figure out what the slowness was in running your HTTP.jl server?

Sorry for the slow response here, but I’ve been deep in some refactorings of other packages and am circling back around to open HTTP.jl issues and how we can improve things.

I ran some simple “hello world” benchmarks and things seem to be on par with a default node server at least:

`ab -n 10000 -c 100 http://0.0.0.0:8086/`  
node

```julia
Percentage of the requests served within a certain time (ms)
  50% 31
  66% 35
  75% 37
  80% 39
  90% 43
  95% 45
  98% 48
  99% 182
 100% 295 (longest request)

```

julia via HTTP.jl

```julia
Percentage of the requests served within a certain time (ms)
  50% 39
  66% 48
  75% 50
  80% 52
  90% 55
  95% 58
  98% 87
  99% 159
 100% 179 (longest request)

```

julia via HTTP.jl using a `HTTP.Router` handler

```julia
Percentage of the requests served within a certain time (ms)
  50% 44
  66% 49
  75% 54
  80% 55
  90% 57
  95% 58
  98% 59
  99% 60
 100% 62 (longest request)

```

`ab -n 10000 -c 1000 http://0.0.0.0:8086`  
node

```julia
Percentage of the requests served within a certain time (ms)
  50% 81
  66% 158
  75% 323
  80% 330
  90% 3722
  95% 8698
  98% 13259
  99% 13420
 100% 25887 (longest request)

```

julia with simple `HTTP.Router`

```julia
Percentage of the requests served within a certain time (ms)
  50% 64
  66% 75
  75% 79
  80% 85
  90% 1574
  95% 7976
  98% 14630
  99% 21373
 100% 21516 (longest request)

```

If you can share, even if just privately an example of the slowness you’re seeing, I’d love to dive in and figure out what’s going on. Happy to jump on a zoom call if that would be helpful.

@yoh-meyers, (and others), I’d love to invite collaboration on HTTP.jl. Julia has a great history of collaborating on core packages to make them best in class. If you have ideas on how to improve things, I’m more than willing to bring on additional maintainers with HTTP.jl. I have a list myself of 5-10 projects I want to work on, and I’m trying to find ways to encourage others to pitch in on developing/helping out.

Would there be people interested in doing something like a weekly (or biweekly) call to discuss progress on web development topics in Julia? I’m happy to host and help put together an agenda if that would help people get a better feel for where work needs to happen.

---

<div class="post-metadata">

**Author:** ![yoh-meyers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoh-meyers/32/14700_2.png) [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)\
**Post date:** [October 10, 2020, 10:59am UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/30 "2020-10-10T10:59:22Z")

</div>

No problem for late response @quinnj.  
Is impressive that you already maintain 10 projects.

I would be happy to be part of the HTTP.jl maintainer team, at least to help it become faster.  
As myself I would like to have a better and more modular MVC web framework out there.

Can DM me and we can arrange an initial call.

---

<div class="post-metadata">

**Author:** ![ImreSamu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/imresamu/32/20677_2.png) [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)\
**Post date:** [December 16, 2020, 3:24pm UTC](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281/31 "2020-12-16T15:24:23Z")

</div>

> [@xiaodai](#):
>
> I can’t find Julia in this benchmark [https://www.techempower.com/benchmarks/](https://www.techempower.com/benchmarks/)

the first http.jl test has been added to the Techempower test;  
( and the final PR deadline 12/28 ; if anyone wants some tuning )

- [TechEmpower/FrameworkBenchmarks R20 deadline 12/28 + first julia "JSON serialization" results](https://discourse.julialang.org/t/techempower-frameworkbenchmarks-r20-deadline-12-28-first-julia-json-serialization-results/51936)

[Previous page](https://discourse.julialang.org/t/http-jl-doesnt-seem-to-be-good-at-handling-over-1k-concurrent-requests-in-comparison-to-an-alternative-in-python/38281.md?page=1)