This has been discussed several times before, but both julia and emscripten have had some changes recently and I wanted to check in to see if anyone has made progress.
I’ve been playing around with this recently in Docker using something like:
FROM ubuntu:16.04
# Install packages needed for building Julia and using emscripten
RUN apt-get update \
&& apt-get install -y build-essential gcc g++ cmake curl gfortran git m4 python xz-utils bzip2 nodejs \
&& ln -s nodejs /usr/bin/node
# Build julia
RUN cd /opt \
&& git clone https://github.com/JuliaLang/julia.git --branch v0.6.0 \
&& cd julia \
&& make -j8 \
&& ln -s /opt/julia/julia /usr/bin/julia
# Get the most recent emsdk
ADD https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz /emsdk-portable.tar.gz
# Set up emscripten -- use 1.37.6 to match LLVM versions with Julia
RUN tar -xvzf emsdk-portable.tar.gz \
&& cd /emsdk-portable \
&& ./emsdk update \
&& ./emsdk install sdk-1.37.6-64bit \
&& ./emsdk activate sdk-1.37.6-64bit \
&& /bin/bash /emsdk-portable/emsdk_env.sh \
&& find /emsdk-portable -name "llvm-nm" -exec ln -s {} /usr/bin/llvm-nm ";" \
&& find /emsdk-portable -name "emcc" -exec ln -s {} /usr/bin/emcc ";" \
&& find /emsdk-portable -name "emsdk" -exec ln -s {} /usr/bin/emsdk ";"
# Build a sysimage as llvm bitcode
RUN cd /opt/julia/base \
&& julia --output-ji inference.ji coreimg.jl \
&& julia --output-bc sys.bc --sysimage inference.ji --startup-file=no sysimg.jl
(This is available here but not optimized for size).
I’m still trying to sort out the correct way to pass flags like ‘-m32’ to the julia build, so this is still targeting x86_64.
So far I haven’t been able to actually export any functions to js – whenever I run something like emcc -v sys.bc -o out.js -s EXPORTED_FUNCTIONS="['_julia_abs_26802']" I get a “function requested to be exported, but not implemented:” error.
Does anyone have updates or progress that they can share?
Is it still worth it to target JS, meaning asm.js, over WebAssembly that is now stable and in the major browers?
Rust has tier 2 support for both by now (both for only “std”):
asmjs-unknown-emscripten ✓ asm.js via Emscripten
[…]
wasm32-unknown-emscripten ✓ WebAssembly via Emscripten
I believe they targeted asm.js before WebAssembly (and they first to support the latter after C/C++ you get from Emscripten).
Version 1.14.0 (2016-12-22)
[…]
This release includes experimental support for WebAssembly, via the wasm32-unknown-emscripten target. This target is known to have major defects. Please test, report, and fix.
They might have different priorities, as they do not have a GC or e.g. BLAS dependency… WebAssembly doesn’t support GC (in the first version), but it seems we could just compile ours to work in WebAssembly.
Another factor is if you want to interopt with JS code.
[ I see they also have other [unrelated] interesting targets 16-bit (or 20-bit variant?) TI MSP430 - Wikipedia and "Preliminary Solaris/SPARCv9 ]
Doing this with Docker is a great idea. That’ll help with keeping versions of LLVM the same, allow better reproducibility, and most importantly document what you tried (every time I try this, I can’t always remember what I’ve tried before ). For the -m32 issue, you could try installing the generic 32-bit Linux version of Julia. That might help to generate more appropriate bitcode. If the 32-bit version of Julia doesn’t work in the 64-bit container, a 32-bit container probably would (of course, that might compilicate the emscripten install).
On the “function requested…” error, that’s what I got the last time I tried this. Something must have changed in how Julia stores/exposes methods. You might try putting the following into a base/userimg.jl:
With luck, that might get myabs() exposed, so you can call it from JavaScript.
As far as emscripten and WebAssembly, emscripten is still the best way to generate WebAssembly. The Rust community has been working at ways to generate WebAssembly without emscripten. That would be really nice, because you wouldn’t need another version of Clang/LLVM. But, it looks like it’s still a ways off, and emscripten provides a lot of necessary work like linking and file I/O. The following post describes how to do a conversion manually that doesn’t involve emscripten (this whole thread is an interesting discussion of the state of LLVM as it relates to WebAssembly):
@Palli – that’s a good point. I’ve been targeting asm.js on the assumption that webasm would have the same issues (and it’s easy to inspect the js output) but I’ll try it both ways. My eventual goal is to use webasm.
@tshort – I’ll try @Base.ccallable and building 32 bit julia and let you know if I make any progress.
Hi! I was just trying to look into this as well, but for some reason, I can’t get the Docker julia to build… Which is surprising cause I thought reproducibility was like a big part of the point of Docker.
Tom Short has made a lot of progress on this since my post above, and some of the furthest progress can be seen in the recent commits here: https://github.com/tshort/jl2js-dock
I’m also working on packaging it up into a web-based application for people to play with here https://github.com/amellnik/Julia-to-JS but there’s quite a bit more work that needs to be done (which I’m planning to do in the next week or so). Best,
Also, fwiw, i reran the docker build without the parallelism (so I could get a better error message) and it just worked … so who knows. I’m excited to read through this new stuff though! Thanks!
I’ve tried a few different routes at generating LLVM bitcode for use with Emscripten:
Julia command-line args – This method is slow, and the code it produces only works for simple Julia code. There is also a problem with naming and exporting of the functions created. The advantage of this method is that it outputs fairly complete code.
CUDAnative approach – The experimental ExportWebAssembly package uses code from CUDAnative to produce bitcode on the fly. It works great for simple code (type stable with no use of libjulia or other C functions). Math code that uses immutable structs and/or StaticArrays should work well with this approach. The main downside is that it is so limited–no arrays, no strings, etc. I looked for ways to try to convert ccall’s to llvmcall’s, but I didn’t find a way to do that. Another issue is that the code generated has some baggage for garbage collection (address spaces in particular) that may not work with Emscripten
Extended CUDAnative approach – It may be possible to substitute out libjulia-type functions with ones that work in JavaScript/WebAssembly. For example, I made a custom array type here. That idea could be built up. One would have to intercept calls to functions that make arrays or do IO. I think CUDAnative is planning to use Cassette to do something like this.
Custom code generation – I’ve started an experimental CodeGen package that uses LLVM.jl to directly generate bitcode based on code_typed Julia code. There is a branch of ExportWebAsssembly that uses this. This can use libjulia functions (also compiled to LLVM bitcode), so a wider range of Julia code may work. The good news is that I’ve gotten some decently complex code to run, including some array creation and manipulation. The bad news is that there are still a lot of gaps that need to be filled in here to be able to run Julia code in general. In the tests, you can get an idea of what works and what is still broken. I think this approach will still probably need Docker or at least some setup with a 32-bit version of Julia. This method offers the most control over code generation.
The bottom line is that it’s going to take a good bit more work to be able to compile code with Emscripten. The last method seems most promising, but it may be tough to match Julia’s builtin code generation (it is quite complex).
This is really interesting. I’ll spend some time going through these links, thanks. Yeah, this is a much harder problem than I originally realized, I was thinking we could just emconfigure make julia, and then run the resultant javascript in a browser.
But yeah, this is really interesting. #4 sounds promising indeed.
I also see there “Emscripten can’t run functions called with ccall”
That’s not very surprising, but not having looked into all the options, I just note WebAssembly is now supported “in all the major web browsers” (i.e. Safari, and finally Edge, but yes not IE).
It should solve the ccall problem (ang Emscripten not needed).
At least this could work for Julia only (or plus C/Python etc.) code, but I believe not still enough for JS interop (if that’s why you want to compile to use JS or its GC, Emscripten may be needed or or some workaround available to interop with JS?).
). For the 
