# Evaluation of a Julia web server for 10k concurrent connections

**URL:** <https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461>\
**Category:** Community\
**Tags:** announcement\
**Created:** [September 25, 2018, 9:31am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461 "2018-09-25T09:31:53Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![ajaym](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@ajaym](https://discourse.julialang.org/u/ajaym)\
**Post date:** [September 25, 2018, 9:31am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/1 "2018-09-25T09:31:53Z")

</div>

In order to evaluate Julia for the server side I ran a few tests. I hope you enjoy reading and do comment.

[https://github.com/aj-monk/C10k.jl](https://github.com/aj-monk/C10k.jl)

---

<div class="post-metadata">

**Author:** ![ImreSamu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/imresamu/32/20677_2.png) [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)\
**Post date:** [September 25, 2018, 11:31am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/2 "2018-09-25T11:31:58Z")

</div>

Thank you for the test 🙂

* * *

related and maybe some reader will be interested.  
As I see: Julia is missing from the popular [TechEmpower Framework Benchmarks](https://www.techempower.com/benchmarks/) results.

see the current languages:

- [https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks](https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks)

And your test is similar than TechEmpower: **“Test type 6: Plaintext”**  
_"_  
_1. The recommended URI is **/plaintext**._  
_2. The response content type must be set to `text/plain` ._  
_3. The response body must be `Hello, World!` ._  
_…_  
_11. The request handler will be exercised at 256, 1024, 4096, and **16,384 concurrency.** _  
_"_

so maybe it is a good candidate for **[FrameworkBenchmarks](https://github.com/TechEmpower/FrameworkBenchmarks)**

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [September 25, 2018, 12:17pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/3 "2018-09-25T12:17:04Z")

</div>

Does anybody know if HTTP.jl’s server is multithreaded already or the numbers are just for a single core?

---

<div class="post-metadata">

**Author:** ![NiclasMattsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/niclasmattsson/32/21988_2.png) [@NiclasMattsson](https://discourse.julialang.org/u/NiclasMattsson)\
**Post date:** [September 25, 2018, 2:32pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/4 "2018-09-25T14:32:15Z")

</div>

And here I am making an online climate model with a Julia back end, and worrying about what would happen if I got 5 users at once. I’m such a noob. 😳

---

<div class="post-metadata">

**Author:** ![Steven\_Sagaert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steven_sagaert/32/29578_2.png) [@Steven\_Sagaert](https://discourse.julialang.org/u/Steven_Sagaert)\
**Post date:** [September 25, 2018, 3:25pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/5 "2018-09-25T15:25:57Z")

</div>

Of course just sending “hello world” isn’t doing much so 5000 concurrent requests of that isn’t that impressive. I’ve implemented a simple recommender system and I’ve load tested it and it could also handle a high load (can’t remember the exact numbers) while actually doing some real work (mostly indexing big matrices and sorting the resulting column).

I see you do an explicit @async but I’m not sure if that is actually needed here for async processing since it’s all io anyway and that is handled (I think) via non-blocking io by libuv. My webserver (via HttpServer.jl) just did that and it scaled just fine.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [September 25, 2018, 5:00pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/6 "2018-09-25T17:00:32Z")

</div>

Nice analysis! It would be great to try to figure out what’s causing the performance degradation and improve it. Getting to 6.8k connections without degradation is a good start.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [September 25, 2018, 5:20pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/7 "2018-09-25T17:20:47Z")

</div>

@quinnj seems pretty close to figuring that out 😉

[https://github.com/JuliaWeb/HTTP.jl/pull/301#issuecomment-424346563](https://github.com/JuliaWeb/HTTP.jl/pull/301#issuecomment-424346563)

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [September 25, 2018, 5:28pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/8 "2018-09-25T17:28:54Z")

</div>

It’s not multi-threaded, nor does it support any kind of multi-threading at the moment, due to Julia’s limited multi-threading abilities. It does, however, support multi-core serving, so multiple julia processes can all listen on the same port and process requests simultaneously (via the `reuseaddr=true` keyword argument, currently limited to unix systems).

An exciting PR to watch is [WIP: parallel task runtime by kpamnany · Pull Request #22631 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/22631), which will allow Julia’s native `Task`s to run concurrently through the use of threads. It’s not entirely clear how the resulting interaction w/ IO (via libuv’s event loop) will play out, but hopefully this would allow _automatic_ better concurrency since HTTP.jl already handles each incoming request in an `@async` block.

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [September 25, 2018, 9:50pm UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/9 "2018-09-25T21:50:43Z")

</div>

So if we start 4 services on 4 cores with `reuseaddr=true` (or load balancer) the a least in theory we get 6.8K \* 4 = 27.2K? Sounds good to me 🙂

---

<div class="post-metadata">

**Author:** ![ajaym](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@ajaym](https://discourse.julialang.org/u/ajaym)\
**Post date:** [September 26, 2018, 7:42am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/10 "2018-09-26T07:42:12Z")

</div>

True, sending in a static page doesn’t **do** much. The intention is to test Julia as a forwarding web server and compare it to popular alternatives out there so that anyone who wants a web front end to something that is already written in Julia need not bother with a non Julia server. Well, at least until it hits 5k concurrent connections.

---

<div class="post-metadata">

**Author:** ![ajaym](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@ajaym](https://discourse.julialang.org/u/ajaym)\
**Post date:** [September 26, 2018, 7:48am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/11 "2018-09-26T07:48:14Z")

</div>

[ImreSamu](https://discourse.julialang.org/u/ImreSamu)  
Thanks for the link!  
Have added to my todo list.

---

<div class="post-metadata">

**Author:** ![ajaym](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@ajaym](https://discourse.julialang.org/u/ajaym)\
**Post date:** [September 26, 2018, 8:02am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/12 "2018-09-26T08:02:00Z")

</div>

The current Julia model is to multiplex **m** tasks on **1** thread (m x 1 model). I have my hopes pinned on [22631](https://github.com/JuliaLang/julia/pull/22631) that will bring in (apart from the parfor interface) an **m x n** model.

---

<div class="post-metadata">

**Author:** ![ajaym](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@ajaym](https://discourse.julialang.org/u/ajaym)\
**Post date:** [September 26, 2018, 8:27am UTC](https://discourse.julialang.org/t/evaluation-of-a-julia-web-server-for-10k-concurrent-connections/15461/13 "2018-09-26T08:27:08Z")

</div>

The @async is simply a way to get back the Julia prompt 🙂. I think HTTP.server.serve() internally spawns a task for each connection, and if I’m not wrong HttpServer.jl does the same.