Hosting Web Apps in Julia

General Usage
,
1

Hey, I am curious as to the current state of web apps in Julia. I have already started playing around with Escher.jl, and was wondering if it would be possible to host such creations online. Does anyone know if this is possible, or if there’s other packages which can do this? And which hosts would this be possible with? Would it be compatible with a standard web server? XSEDE Gateways?

[P.S. For those who are curious, I am thinking about creating a “WolframAlpha for numerical differential equations”, where you punch in a system of ODEs/PDEs, and can get some plots out / can download the numerical solution in an Excel/csv form. With ParameterizedFunctions.jl already able to convert natural syntax to Julia functions, the only thing left is how to make the actual web app, so I think I have to give it a try!]

9 Likes
2

I do not think you want to use Escher.jl. It’s a bold effort, but it’s not ready for practical use. Genie.jl is an even bolder effort, but it’s also not ready for production, and I’m not sure its approach makes sense here.

I think you want a bare-bones API back-end that handles the Julia bit, then do everything else in a normal front-end framework like Angular, React or similar.

For the back end you could definitely do it in HttpServer.jl or Mux.jl, but I don’t know how they scale. You could also write the back end in Node.js using node-julia to hand information off to Julia. This does work, but node-julia is somewhat painful to use.

The back end would need to be hosted on something that can run Julia, but the front end could be on any vanilla webhost.

This sounds like a pretty interesting project, and if you could use any help on either side let me know. I also have some hosting space that you’re welcome to.

2 Likes
3

Hmm, I was hoping there was something more developed that I could just plug into. I don’t want this to turn into a main project at all or something which is difficult to maintain, and, more importantly, I can wait. In the meantime, if anyone develops something that can do this and wants a test case, please let me know and I’ll give it a whirl.

1 Like
4

I think Escher would be a good option here. You can host it on any web server in principle. Cc @shashi

5

Can you expand on this a little? For most of what I want to do Escher.jl already works. I am having trouble getting it to display graphics (even for the examples), but I am sure that will get fixed up, in which case it will do everything I need it to do. What I don’t know how to do is get it hosted / properly allocate resources.

1 Like
6

Disclaimer: I’m not much of a Julia- or web developer, so I hope the more knowledgeable can correct me if I get anything wrong here.

I don’t mean to disparage the efforts of @shashi and others – Escher is a great project and does a lot of cool stuff. However, I agree with its readme in that it’s not ready for public projects, and I also think the approach it takes may not be a good fit here.

What you want is a website that that lets the user solve differential equations. For this, you need Julia since trying to solve differential equations with javascript is the stuff of nightmares. However, you don’t need Julia to give you a website – websites are a known problem which have known solutions. Escher uses Julia to give you the website, but this comes at a cost.

First, whenever you interact with a component on an Escher page in a non-trivial way, you have to communicate with the back end. This makes the page less responsive and also takes a surprising amount of data. For example, on the Serpinski demo, dragging the slider back and forth a few times exchanges half a MB of data with the back end.

846×1219 45 KB

On the boids.jl example, running the simulation uses 3 MB/sec of bandwidth.

This doesn’t make a difference for me running a single instance of the webpage on localhost, but it won’t easily scale to the case where you have multiple people, or even a single person on a low-bandwidth connection.

Always having to shuffle data back and forth between the client and the server also prevents you from easily doing a lot of things that are natural parts of modern web pages. For example, you might want to plot the solution to the differential equation. If you want to let the user zoom in on the y-axis of a Gadfly plot, you need to implement custom controls for doing this. Then, every time the user touches one of those controls, the page needs to talk to the server, which rerenders the entire plot and sends the whole thing back. However, there’s really no need to use the server for something like this, and it’s trivial with client-based plotting packages like Plotly or Highcharts.

Someone should correct me if this is not correct, but I think Escher is also single-threaded and blocking. If one user submits a differential equation that takes two seconds to solve, no other user will get any response in that time, and all the requests that come in during that time will pile up.

I think the easiest way to do something like this is to have a normal web page served by any old webserver, and use Julia only for the back end. Looking at this a bit more, I think that JuliaWebAPI.jl might be a better choice than the options I mentioned previously. I’m still figuring out exactly how to use it with JuliaBox, but once I have it going I think I can throw this whole thing together pretty quickly. If you have a notebook that shows roughly what sort of diff-eq solver you want to provide send it my way, otherwise I’ll set it up with a simple example.

4 Likes
7

I have similar goals. Long term, I’m hoping to be able to compile Julia to Emscripten/WebAssembly. This would allow one to compile a model that the user could twiddle inputs and see results. It’d all be done in the browser–nothing server side. Rust, another LLVM language, just got mainline support for Emscripten and WebAssembly:

http://www.hellorust.com/emscripten/

I’ve managed to compile DAE systems in Modelica to JavaScript using Emscripten. It runs surprisingly well, so doing the same from Julia would be great. Here’s an example of the end result:

http://distributionhandbook.com/calculators/mdpad.html?UrbanPrimary.md

1 Like
8

@tshort.rlists - Now that’s some exciting stuff. I have no clue what’s involved in compiling arbitrary Julia code (with arbitrary package dependencies?) to Emscripten-friendly c/c++, but I’m looking forward to seeing what people figure out.

9

I’m not sure this is helpful (in general):

Can’t quite believe “Write Rust code in Julia” (the code is short…)

but if it works, Rust → WebAssembly was brought up. Directly from Julia would be cool, but I don’t care how it works behind the scenes if there was a package doing it and hiding how…

“I am curious as to the current state of web apps in Julia”

do you for sure mean, RIA, e.g. client side?

Needs it be Julia? I’m only familiar with names, Django (and web2py, Kivy), and can’t see a reason it wouldn’t work with PyCall.jl.

I just discovered:

through:

meaning:

It’s newer than Escher (that seems to be cool; wasn’t aware of the downsides); I only have/had a high-level view, there was at least one more, non-low-level, I’ve really not looked into…

http://juliawebstack.org/ seems outdated (except for low-level), could it be updated, or at least we have one central place for overview?

http://www.juliabloggers.com/tag/web-stack/

10

I agree this is the way to go. Here’s a notebook which outlines how the Julia side would work. I have the idea set up in a way that when a user clicks a solve button it just sends some strings over to Julia, and Julia sends back Javascript for a Plotly plot and/or arrays of data. As you can see, it’s very simple and the Julia side is already done!

11

See JuliaAPIDemo for a few first steps. All it does at the moment is square a list of numbers. A few open items are:

  • I don’t have the API feature on my JuliaBox.com account yet – I sent an email to them to see if they’ll let me test it. This project includes code for a local one.
  • JuliaWebAPI doesn’t seem to support https, and because I haven’t set up a custom domain for this project yet, the github-pages version of the page (https) can’t talk to the non-https back-end. If you host the front-end locally or elsewhere with http then it works fine.
  • If you want to accept arbitrary input to be parsed by Julia (like function definitions) we’re going to need some serious sanitation so we’re not eval’ing bad stuff.

That being said, I think all the required parts are in place. Once I resolve the minor stuff above we can start putting the actual diff-eq stuff in.

12

There shouldn’t be any function definitions sent, it should just be filling in the ParameterizedFunction @ode_def expression (see the notebook for an example). But yes, to make it safe we can just check to make sure is nothing that does has the function definition syntax at all. I am not an expert on this but I’ll defer to you for whether its safe as is, or whether it needs a little more.

13

Interesting thread; I’ll try out JuliaWebAPI.

There are many other unsafe constructs of course but I used this at some point to detect Julia func defs so it might be a useful start:

expr_has_head(s, h) = false
expr_has_head(e::Expr, h::Symbol) = expr_has_head(e, Symbol[h])
function expr_has_head(e::Expr, vh::Vector{Symbol})
    in(e.head, vh) || any(a -> expr_has_head(a, vh), e.args)
end

has_function_def(s::String) = has_function_def(parse(s; raise=false))
has_function_def(e::Expr) = expr_has_head(e, Symbol[:(->), :function])

str1 = """
function f1(x)
    x+1
end
"""
has_function_def(str1) # => true

str2 = """
f2 = (x) -> x+1
"""
has_function_def(str2) # => true

str3 = """
a = 2 * 3 / x
"""
has_function_def(str3) # => false
2 Likes
14

How far would you say we are from similar capability in Julia? Is it a matter of just dumping our LLVM into emscripten, or will it take more time and work?

15

@robertfeldt – This is very handy – thanks!

@ChrisRackauckas – For the next pass I’ll recreate the example in the notebook you sent and use Robert’s snippet to disallow any function heads or definitions in the di’s. In the future, maybe we can figure out a whitelist of function heads that could be used (things like sin, ln, etc). We’ll still need to come up with a way to deal with definitions that are too costly to evaluate.

I’m also starting to think about what a good production back-end would be. I’m going to talk to the Julia Computing people later this week to see what they do for JuliaRun (the API stuff is no longer under JuliaBox). If we don’t anticipate any need to scale I could swing for a cheap EC2 instance. A better option might be to use something like Amazon Lambda, but it doesn’t look like it supports Julia. It might work with Heroku though.

16

Maybe we could have this be an advert for JuliaRun :wink:.

17

Getting the full Julia working (REPL, libraries, test suite, etc.) in Emscripten/WebAssembly would be a lot of work. Ultimately, it should be doable because Julia is well matched for this. A more realistic starting point is to statically compile smaller chunks of code. This should be easier given the work Jameson/Julia Computing has done in this area.

I’ve tried it in the past. I could get simple functions compiled and run in the browser. These functions would involve basic types like Int’s or Float64’s. I couldn’t get arrays or indexing to work at all, and there were some other Emscripten issues I didn’t know how to deal with. As a starting point, I thing you need to consider the following:

  • Match up the versions of LLVM that Julia and Emscripten use
  • Statically compile Julia code to 32 bit LLVM output (-m32)
  • Compile the C code that makes up the Julia base library
  • Link them together–may need to handle naming somehow

At one point, Rust compilation used special LLVM passes. I don’t know if that’s needed anymore.

1 Like
18

I am curious whether Sam O’Connor’s AWSLambda.jl would work for you.

19

Jeez, there’s clearly a lot of amazing Julia wizardry going on that I’m not aware of. That would definitely do the trick. Thanks!

20

Thanks.

I’d just be happy compiling a small escher web app with a server side backend. Can that be done now (or in 0.6 when hopefully c libs move out of base)?