Hey everyone, I’m exploring UI tools and integrating them with Julia. As part of this exercise I ended up making a nice UI in JAVAFX, now I need to hook it into Julia…
So there’s two ways to go about this from what I understand(if there are others please suggest!).
have a server where JAVA sends instructions to be understood by Julia (zmq, webio, whatever).
have JAVA natively call Julia code and use some kind of wrapper (https://github.com/rssdev10/julia4j - probably outdated by now) and interpret the data via JAVA. This sounds like a pretty serious effort, if the machinery isn’t in a working state.
Okay, so I am working on a pretty data intensive idea. A lot of manipulations are happening. So most operations need to be flexible/generic. Constantly calling down to julia, retrieving, and sending back data is gonna bite me in the butt really quick. Having to write hyperspecific wrappers will also kill my productivity.
Is it common for people to write pseduocode/enums/hashmap API’s for interfacing languages? IE:
JAVA sends : \x23\x26\x12 (hunk of data)
Julia receives: \x23, \x26, \x12 and maps it to julia functions A, B, and C. Then composes the three functions (foldl, or whatever on the tuple), receives the data, applies the composite operations. It then sends the transformed data back?
I see this level of wrapping as very laborious for many operations. Especially if there are a variety of parameters for each function. Then again, a decently generic framework could maybe be made if the number of params and types of each call can be introspected in Julia…
That being said… Going this route could mean that embedding HTML in say a JAVA app could lead to really nice use of Julia code with minimal effort. (Call plot, spin up connection, Java connects, and displays Julia plot).
What kind of design patterns do people typically use to interface languages like these? I know the JS crowd does this sort of stuff all day.
Alright… I had an idea… The ZMQ interface could be really easy. So Julia is Lisp-like, we have meta programming, it’s not a weak feature of the language either. Although I try to avoid it as much as possible I think it could work…
Here’s my current plan…
Java kicks off a background Julia instance.
Java listens over ZMQ for Julia, they handshake yadda yadda
Julia has one main listening function - yep… Java sends strings of julia code over ZMQ, to Julia which handles all I/O, and it looks like this…
It seems hacky, but… Julia literally handles Julia code. The tricky bit will just be getting everything standing up. Not sure how slow this will be, my guess is it won’t be fast… I used to do this sort of thing in JS way back in the day with ye olde eval().
What you’re doing sounds pretty cool. I’ll follow along to see how it turns out. Since you mention ZMQ and an attached Julis instance. Maybe it’s be worth looking at how Jupyter communicates with IJulia. I understand they use ZMQ for communication. Here’s a thread discussing running a remote REPL via IJulia which is almost what you want:
@venuur thanks for following along. It’s kind of a scary place to be but its also kind of fun! I am really brushing up on some concepts that were dormont in my brain for a while.
So I made a minimum viable example that requires JeroMQ to be built with maven(don’t run the tests trust me its fine and it’ll save you a half hour), and linked to the JAVA project(https://github.com/zeromq/jeromq/releases/tag/v0.5.1). Also need the ZMQ package to be installed in Julia! What its doing…
Java calls a Julia program to be run in the background
Java and Julia exchange a classic syn-ack-synack handshake (useful for debugging)
Java zmq’s to julia the julia function it wants to use
Julia evaluates the java instruction, applies it to a float64, then sends it back to java.
Java converts the number from little endian to big endian then displays it.
They both close the connections and the world is at peace
import org.zeromq.ZMQ;
import org.zeromq.ZContext;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
public class ZMQBareBones {
Process juliaServer;
BufferedReader stdInput;
BufferedReader stdError;
public static void main(String[] args) throws Exception {
ZMQBareBones zmqApp = new ZMQBareBones();
zmqApp.run();
}
public void run() throws Exception {
String s = null;
try {
juliaServer = Runtime.getRuntime().exec("julia /home/caseykneale/Desktop/whater.jl");
stdInput = new BufferedReader(new InputStreamReader(juliaServer.getInputStream()));
stdError = new BufferedReader(new InputStreamReader(juliaServer.getErrorStream()));
// read the output from the Julia console directly into java!
//Currently not working needs a loop/wait/cancel routine but whatever the concept does work.
System.out.println("Communication Log:\n");
if ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
System.out.println("Beginning ZMQ:\n");
try (ZContext context = new ZContext()) {
// Socket to talk to clients
ZMQ.Socket socket = context.createSocket(ZMQ.REP);
socket.bind("tcp://*:5555");
boolean handshake = false;
double result = Double.NaN;
while ( (!Thread.currentThread().isInterrupted()) ) {
// Block until a message is received
byte[] reply = socket.recv(0);
if(handshake == false) {
String replymsg = new String(reply, ZMQ.CHARSET);
if (replymsg.equals("SYN")) {
System.out.println("Received: Communication from Julia Client.");
String response = "ACK";
socket.send(response.getBytes(ZMQ.CHARSET), 0);
}
if (replymsg.equals("SYNACK")) {
System.out.println("ZMQ Handshake completed!");
handshake = true;
socket.setReceiveBufferSize(8);
System.out.println("Sending fn to Julia for evaluation...");
//Now that we know we have communication back and forth... let's do something dastardly...
socket.send("exp_decay".getBytes(ZMQ.CHARSET), 0);
//socket.send("reciprocal".getBytes(ZMQ.CHARSET), 0);
}
} else {
System.out.println(reply.length);
//Java likes big endian, Julia likes little endian - such is life...
result = ByteBuffer.wrap(reply).order(ByteOrder.LITTLE_ENDIAN).getDouble();
System.out.println("Computed result: ");
System.out.println(result);
}
}
}
//Close Julia server
juliaServer.destroy();
System.exit(0);
} catch (IOException e) {
e.printStackTrace();
System.exit(-1);
}
}
}
println("Execution of Julia code successful.\n Starting compute client...")
#define some inhouse fns for our "library"
exp_decay(x) = exp(-x)
reciprocal(x) = 1 / x
using ZMQ
client_socket = Socket(REQ)
connect(client_socket, "tcp://localhost:5555")
send(client_socket, "SYN")
msgback = recv(client_socket, String)
if ( msgback == "ACK" )
send(client_socket, "SYNACK")
end
awesomest_data_ever = 111.111
#get the Java user's desired function call
function_call = recv(client_socket, String)
result = eval( Meta.parse("$function_call($awesomest_data_ever)") )
#pass back the result as a Float64
send(client_socket, result)
#we're done here...
close(client_socket)
What I’m wondering now is - is it easier to just use a BufferedWriter to write to a background Julia REPL or something? Not sure…
Also the pattern of a massive ZMQ loop is pretty impractical for an interactive application. Might need to see how other people are doing this. My guess is they have listeners polling 24/7 and they drop in and drop out of the server context regularly.
Implementing the Jupyter ZMQ protocol from scratch was more than I wanted so I used a python template. The custom Python kernel started a Chicken process that listened on a socket for strings terminated by an end of file marker. The Chicken server did a parse-eval just like you are doing.
It was however missing any security or authentication. Also it needed to be a bit more robust to malformed strings.
Overall it was a good learning experience on sockets. It felt pretty snappy to me but my data was small. I guess in an interactive application you’d have a dedicated thread communicating with the Julia process and the main event loop would check for events posted by that thread.
The xeus library implements a Jupyter kernel “framework” in C++, in which you can create a kernel for a new language by sub-classing and implementing a few methods for eval, display, etc:
@venuur - forgive me but I have no idea what Chicken is. Googling “Chicken Kernel” lead me to some interesting recipes though :P. “Jupyter Chicken” was a little more fruitful and a little less deepfried. Yes, right now there is zero security whatsoever, and someone could easily inject malicious Julia over ZMQ. Or I suppose JAVA the other way.
Good call on the multithreading. Yea, there’d have to be multithreadedness I think to handle this in a reasonable way.
@ihnorton - This looks pretty darn fancy. So I am supposing the suggestion here is - write a julia kernel for zeus, use JNI(or something) to call down to Xeus, and access the kernel that way? I could see a usable interop interface happening that way. Sounds like a lot of work though?
Hmmm Hmm Hmmm…
A somewhat more insane way to handle this is to use Java to literally write a Julia program, that it then writes to disk, and executes… But I’d rather not go that far down the rabbit hole and all computing would have to be done with lots of I/O exchanges (ew).
Ah, sorry, that suggestion was in response to the subject of writing a kernel for other languages: xeus can help/save significant time there, if you don’t mind the C++ dependency. As far as the original question, you’ve listed the two most tractable options in order of difficulty: network/RPC, and a JNI wrapper of the Julia embedding API. (in theory there’s also the JRuby approach, but that involves considerably more effort than using the JNI/embedding)
Right now I’m taking a break from splicing together code to call Julia from Java - my needs are less oriented toward UI than you I think. I have a boundary condition where the primary software system is in Java (and will remain so for good reasons), but I want to take advantage of Julia for some computational tasks where there are not good solutions available in Java.
I have a template put together to use JNI to call C wrappers to the Julia code. I think it will be pretty easy to generalize.
Would you be able to share the JNI to C wrappers? Or potentially want some assistance there? I think that could be the easiest way, but, I’m not sure exactly how much nuance there would be along the way.
The reason I steered away from it is, C as a dependency can be painful for multiplatform.
I do need to clean them up a little, but then I’m happy to share. Technically I’m on holiday now for US Thanksgiving, so it may not be today…
I should note too that I searched around here on discourse and other places, and it wasn’t clear that the information I found on working with Java/Julia was current or that it would work well, so I’ve been relieved so far!
For me, the code must run on Linux, but I do like to prototype/test on Mac OS X. There are some tricks to deal with in naming of dynamic libraries, etc. but these are not too hard to work with. So, for me I’m not too worried about multiplatform issues at this point.
@avik - thanks for that link. I somehow missed that link in the docs.
Are there any good patterns for securing eval? Maybe make sure any function called exists in the namespace of the requisite Julia project? Make a “safe_eval” function or something.
I cleaned up my test of call Julia from Java JNI, and the sample code is at: https://github.com/lwhitefox/JavaJNI2Julia.git
It’s mostly cleaned up and should be pretty clear, though probably not optimal in several ways. There are two Java functions, one getting only singular values from a small test matrix, the other generating a Java object that has full SVD results for the same test matrix.
Awesome example! I’ll have to grok it a bit. I’m starting to think ZMQ is easier, and removes at least one more language from the mix.
But I did find something very promising for hucking big chunks of data across languages:
So I know Arrow is kind of old-hat - but - there seems to be a good use case here for data intensive operations…
Java → Sends fn to Julia over ZMQ → Julia executes fn on in memory data → streams Arrow to Java
Now the question burning in my mind is this… Have the Julia wizards worked out streaming Arrow so that no copies are made like has been done between JVM & C, JVM & Python, etc? Or should I just ferry results from Julia to Java leaving the heavy lifting in one place? Decisions decisions…
I’ve not used ZMQ, but with a quick look at some resources it does seem to be appealing - have you gone further with testing it to implement any significant computation examples?
For me, the JNI/C approach to embedding Julia is conceptually easy though a bit tedious and brute force. I like the idea of being able to send messages to a running process, but I am much less familiar with the kind of coding.
I didn’t see too much documentation for the Julia ZMQ package either; it would be great to have more examples.
Yea it’s not very documented, and it’s not the most intuitive thing either. Its one of those classic Julia packages where it is effectively the same as it’s C/C++/Java counterparts so they expect you know those, and he rest is intuitive… Solo dev didn’t have time to write docs - I empathize… I think it’s just a battle of what design decisions to make?
Pass lots of data back and forth? Probably want ZMQ or Arrow or something similar.
Pass lots of commands back and forth:
a. JNI + C to Julia with wrappers and done
b. Java + ZMQ to Julia with wrappers and done
c. Java + ZMQ to Julia with eval/metaprogramming + security holes to fix
d. (update) Java launch julia process in interactive mode, call API directly via output writer. Then read in results via buffered reader.
I could explore the ZMQ world a bit more, I haven’t passed any big vectors/arrays back and forth yet with Java, but I did help in a project that did so with Julia and C++.
I am slowly building a GUI to get a feel for what functionality I want at the UI level. Then trying to decide how to architect the actual software aspect of the thing. It’s taking a while because I am working a day job and signed up for too many other things in my free time. I will definitely flesh out some ZMQ examples. Also, the UI I’m cooking up is pretty weird, some novelish things going on so it’s taking longer then I wanted it too. Anything I discover I’ll be sharing. Might be worth making a repo or contributing to your repo with some examples and let people improve them.
I like your solution, it’s clever and pragmatic. I’m just lazy and deciding what corners I can cut/break to get the same functional result. Solo Dev is it’s own ball game…
If you get cool examples with ZMQ, it would be great I you could share. I am completely missing the previous experience with ZMQ you mention so it is a bit a mystery to me at this point!
public class CrazyJuliaInterop extends Application {
private Scene scene;
private Process juliaServer;
BufferedReader stdInput;
BufferedReader stdError;
@Override public void start(Stage stage) throws IOException, InterruptedException {
stage.setTitle("Web View");
String s = null;
juliaServer = Runtime.getRuntime().exec("julia -i /home/caseykneale/Desktop/JuliaInterop/FrontEnd.jl");
stdInput = new BufferedReader(new InputStreamReader(juliaServer.getInputStream()));
stdError = new BufferedReader(new InputStreamReader(juliaServer.getErrorStream()));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(juliaServer.getOutputStream()));
System.out.println("Waiting for Communication from Host:");
if ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
writer.write("println(\" REPL is working! \")" + "\n");
writer.flush();
System.out.println("Julia REPL says:\n");
if ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
scene = new Scene(new Browser(),750,500, Color.web("#666970"));
stage.setScene(scene);
stage.show();
}
public static void main(String[] args) {
launch(args);
}
}
class Browser extends Region {
final WebView browser = new WebView();
final WebEngine webEngine = browser.getEngine();
public Browser() {
getStyleClass().add("browser");
webEngine.load("http://localhost:5570/");
getChildren().add(browser);
}
}
So what does this do…
Interop #1:
Java Opens the Julia Script and connects to it via a web browser element(JavaFX required).
Interop #2:
Then because Julia was called in interactive mode -i we can read and write from a hidden REPL. No ZMQ, eval, or secret sauce required.
This is probably the “easiest” way to get julia interactivity… I still don’t like it, but it’s something to illustrate as possible. Worth noting that the web browser thing is overkill and unneccesary for most sane applications, but - in spirit of this thread I had to try it and see if/how it worked.
It seems the biggest advantage to ZMQ, and WebIO based interops are there isn’t any background precompilation going on except once(when the program loads).
