HTTP.jl doesn't seem to be good at handling over 1k concurrent requests, in comparison to an alternative in Python?

Specific Domains Web Stack
21

Is some time since discussion, but curious what thoughts are?

Basically most of the popular “fast” web frameworks use http-parser (a C library).
List includes Bjoern, Vibora or NodeJS, among others.

So there are 2 solutions.

1 - Use ccall to use http-parser for request handling. This is not ideal, as it will beat fact that Julia is supposed to be equivalently performant to C. Still, writing back to incoming connection functionality will need to be implemented (can rely on HTTP.jl).
2 - Implement logic freshly in Julia. Having done some research, have seen that PicoHTTPParser is faster than http-parser. Again, writing back to incoming connection functionality will need to be implemented (can rely on HTTP.jl).

Above mentioned, would recommend 2nd option.
This would really help to promote Julia’s attractiveness!

What do you think?
I have started translating PicoHTTPParser to Julia

2 Likes
22

FYI: There’s an (outdated) Julia wrapper for the/a C http-parser library:

It predates the JLL/artifact system (and Julia 1.0), which I think changes things. I see no good reason to avoid good (debugged) C code.

It’s good to have options if you want to port (or wrap) PicoHTTPParser, but I think we should concentrate on the future HTTP/3 (and HTTP/2). There are good C libraries for, and this is a big change, and unfeasible to port I think, as the libraries are big. I suggested at some point adding to HTTP.jl but issue was closed as “HTTP.jl is for pure-Julia code only” (I’m not sure too wise, given JLL possibility, at least a competing package could do it):

23

Hello Páll!

I agree with you on focusing on the future (HTTP3).
Would be more rewarding for the effort.

Only reason I suggested translating PicoHTTPParser, is to have less dependencies.
See that http-parser is indeed a translation, only to be depreciated.

Pity to see that HTTP.jl is Julia-only.
It is a good library, but I have in mind a “server-only” lightweight and fast alternative (HTTP.jl includes making requests etc, which is not needed for most simple API server applications).

Really have in mind something fast, so that running an API server in Julia, starts to attract non-Julia web coders.
So if you are open for collaboration, would be happy to start coding an alternative web server library with you.

3 Likes
24

Julia certainly seems capable of being fast, so I don’t see why HTTP.jl should be slower than other languages. I do realize the stop the world GC could be an issue if latency needs to be strict.

Has anyone tried profiling HTTP.jl? Where was the time being spent?

1 Like
25

In theory yes. But parsers in other languages use optimisations, such as:

  • predicting content of buffer
  • better use of underlying hardware

Julia is capable of both of these mentioned points.
But having quickly parsed the HTTP.jl code yesterday, seems this is not case, though buffer is parsed in chunks (and not char by char).

Also I don’t find HTTP.jl code base well structured, at first glance.

So to answer your point, yes, though a substantial refactoring would be necessary

1 Like
26

I’m kind of disappointed with the last posts. I don’t want to come off hostile, but Julia is a new language and we are lucky someone wrote an HTTP server, but you are complaining because the single/first HTTP server that was written natively for Julia isn’t the fastest/best around. I feel like your statements mean that if we can’t build the fastest/best implementation around then why even bother.

Do you feel that the first HTTP server written for C was the fastest/best implementation around? Why do you feel that the first implementation written in Julia should compete with the best implementations C has to offer?

Truthfully when I develop new code I try to be conscious of of performance but my goal is really to make the code bug free with good abstractions and structure. Then if the performance isn’t doing it, I profile and optimize the bottlenecks. As you apply optimizations the code often becomes harder to read and more bug prone. So having the first implementation that is well structured and readable to debug makes life easier.

If the current implementation of HTTP doesn’t meet your needs then please, please, please start a new project to write a better one, or contribute to the current one to improve it’s speed.

3 Likes
27

I don’t think they are necessarily criticizing anything, just noting what should be made better, for anyone interested.

I’d vouch for writing the interface completely in Julia (as HTTP.jl is) instead of C. Julia code certainly can be speeded up, while C code hardly can be more extensible and accessable.

2 Likes
28

At least from my side, this was intention.
Am openly stating what needs to be improved for Julia to be able to serve heavy Devops applications.

Indeed I even suggested coding an alternative library.
I believe that having a healthy discussion can lead to new ideas and collaborations (hence my posts).
FYI I already started coding a Flask (Python) like web library. And if you look at Github tasks you will see that a faster HTTP parser is part of list.

So in conclusion I am in process of coding an alternative server library in Julia, inspired from PicoHTTPParser .
While coding everything in Julia is optimal, I don’t deny that for the HTTP 2/3 stuff, I might simply import C libraries.

Any feedback/comments are welcome

2 Likes
29

Hi @aviatesk,

Did you ever do more detailed profiling and figure out what the slowness was in running your HTTP.jl server?

Sorry for the slow response here, but I’ve been deep in some refactorings of other packages and am circling back around to open HTTP.jl issues and how we can improve things.

I ran some simple “hello world” benchmarks and things seem to be on par with a default node server at least:

ab -n 10000 -c 100 http://0.0.0.0:8086/
node

Percentage of the requests served within a certain time (ms)
  50%     31
  66%     35
  75%     37
  80%     39
  90%     43
  95%     45
  98%     48
  99%    182
 100%    295 (longest request)

julia via HTTP.jl

Percentage of the requests served within a certain time (ms)
  50%     39
  66%     48
  75%     50
  80%     52
  90%     55
  95%     58
  98%     87
  99%    159
 100%    179 (longest request)

julia via HTTP.jl using a HTTP.Router handler

Percentage of the requests served within a certain time (ms)
  50%     44
  66%     49
  75%     54
  80%     55
  90%     57
  95%     58
  98%     59
  99%     60
 100%     62 (longest request)

ab -n 10000 -c 1000 http://0.0.0.0:8086
node

Percentage of the requests served within a certain time (ms)
  50%     81
  66%    158
  75%    323
  80%    330
  90%   3722
  95%   8698
  98%  13259
  99%  13420
 100%  25887 (longest request)

julia with simple HTTP.Router

Percentage of the requests served within a certain time (ms)
  50%     64
  66%     75
  75%     79
  80%     85
  90%   1574
  95%   7976
  98%  14630
  99%  21373
 100%  21516 (longest request)

If you can share, even if just privately an example of the slowness you’re seeing, I’d love to dive in and figure out what’s going on. Happy to jump on a zoom call if that would be helpful.

@yoh-meyers, (and others), I’d love to invite collaboration on HTTP.jl. Julia has a great history of collaborating on core packages to make them best in class. If you have ideas on how to improve things, I’m more than willing to bring on additional maintainers with HTTP.jl. I have a list myself of 5-10 projects I want to work on, and I’m trying to find ways to encourage others to pitch in on developing/helping out.

Would there be people interested in doing something like a weekly (or biweekly) call to discuss progress on web development topics in Julia? I’m happy to host and help put together an agenda if that would help people get a better feel for where work needs to happen.

16 Likes
30

No problem for late response @quinnj.
Is impressive that you already maintain 10 projects.

I would be happy to be part of the HTTP.jl maintainer team, at least to help it become faster.
As myself I would like to have a better and more modular MVC web framework out there.

Can DM me and we can arrange an initial call.

4 Likes
31

the first http.jl test has been added to the Techempower test;
( and the final PR deadline 12/28 ; if anyone wants some tuning )