Achieving parallel web request handling with Julia (httpserver.jl or other)


#1

I’ve written a web server in Julia using httpserver.jl. For simplicity, let’s say that I have one endpoint that runs a Julia function to crunch some numbers and then return them to the browser. Let’s also assume for sake of discussion that the function takes 60 seconds to execute and complete.

While the first request is being processed, any subsequent requests will sit and spin away waiting for the first one to finish. The browser hangs until the first request completes and then the second one gets processed and returned and so on.

Assuming the machine has enough resources to handle the load, what is the current recommended way to achieve parallel request processing with a Julia web server?

Thanks in advance for the help!


#2

There’s some support in Base for allowing ports to be “reused”; i.e. multiple processes could listen on the same port and requests would be accepted in parallel. I haven’t personally used it yet, and I think support is limited (only works on linux?), but it’s probably the most robust route if you really need a parallel server.

In the HTTP.jl package, which is a rewrite and modernizing of HttpServer.jl, among other things, the Servers package provides quick and easy ways to spin servers up and pass in “handler” functions. When requests are received, they are processed asynchronously using @schedule blocks, so multiple requests can be processed simultaneously.

In one of my own personal applications, I spawn additional julia workers and spawn long-running computations off on them. Typically, http isn’t really meant for long-running/waiting requests like this: you’d probably be better off restructuring the client to make an initial compute request and then periodically poll to see if the computation is finished. You could also consider using websockets for a hybrid approach of persistent connection with the ability to bi-directionally “push” messages.


#3

Just echoing @quinnj response. I have a rule of thumb:

  • If it takes less than 3s, handle it through http
  • Otherwise, schedule a worker

Probably can do this on heroku with a web and worker dyno (it would be free if you got it working)

(the grain of salt is i’ve only hosted using heroku with rails.)


For interesting links check out,


#4

I agree with the above, and would add that it’s possible to schedule workers within a single hosted container if you don’t want to mess with separate worker dynos (or the equivalent in other platforms).


#5

Thank you very much, everyone. I will investigate all of these routes :slight_smile: Cheers.


#6

It was really interesting discussion (and nice work!) year and half ago. You wrote:

I would like to know your opinion about Escher.jl and Genie.jl (or other pure Julia solution) if they are ready now…

Or would you still prefer Angular and if yes then why? Thanks! :slight_smile:


#7

FYI there was also this proposal of a google summer code there https://julialang.org/soc/projects/general.html#middlewares-for-common-web-application-chores-in-muxjl

Middlewares for common web application chores in Mux.jl

Implementation of mid-level features - specifically routing, load-balancing,
cookie/session handling, and authentication - in Mux.jl. The implementation
should be extensible enough to allow easy interfacing with different kinds of
caching, databases or authentication backends.
(See Clojure/Ring for inspiration).

Expected Results: Improvements to the Mux.jl package.

Required Skills: Experience with web development.

Recommended Skills: Knowledge of various web frameworks, especially the design decisions and tradeoffs behind them.

Mentors: Mike Innes

It seems it wasn’t allocated this year in the recently published gsoc 2018 student list
I don’t know if there was (or will be) any others evolutions


#8

As far as I can tell there are still more students to announce themselves. I’m pretty sure we haven’t 27 introductions yet. Maybe the MUX project is still coming.


#9

I have little experience in web development but I’ll be developing a web app in WebIO for this year GSoC and was planning to deploy it with Mux eventually. It’d be interesting to know if Mux is still the recommended way/ how robustly one can use it to deploy apps or otherwise how easy/hard it’d be for WebIO to support the new HTTP.jl framework.


#10

Escher is no longer being developed as a package, although parts of it are being reused. I haven’t used Genie.jl but it doesn’t seem like a good fit for my typical use case.

I usually have some large front-end application that needs fairly complex routing, components, and so on, which is why I use standard tools like Angular/React and existing design frameworks. Occasionally I need to create a new API that uses julia, and for that my use case is very similar to the one Jacob mentioned above.

I’m a big fan of the new HTTP.jl and would recommend it over Mux. HTTP provides a lot of the same functionality for interacting with headers, creating responses, etc, although it doesn’t have a high-level interface for stacking middleware and setting up routes. I set up a bit of this in Joseki.jl but you can also just use HTTP directly.


#11

I’m not sure what are the technical obstacles, but would you consider providing support for WebIO.jl in Joseki? This is the file that defines Mux support for example, not sure how hard it would be to port it. The advantage is that one could then publish all InteractNext based GUIs using your Joseki package.


#12

It would be possible, but Joseki is narrowly focused on API creation so I think it would make more sense to create provider directly with HTTP.


#13

Do you think that it is

  • a) technically possible
  • b) probable

that Julia will solve 2 language problem and allow people to avoid javascript (maybe css too) in creating web apps?

(Btw joseki.jl seems nice! :slight_smile: Do you plan tesuji.jl in the future? :wink: )