I’m researching which language is best to create a physics engine and would like to use Julia. The requirements are simply that I must be able to produce a DLL which can pass custom data types to/from a front end, written in Java, on a windows machine. I’d like to use Julia to show it’s feasible for future development, but am concerned about the complexity of integrating a Julia DLL with Java.
I use Julia for prototyping, but this is my first attempt trying to use it in a production setting. Any advice on the following would be incredibly helpful:
Is integrating Julia into mixed-language projects (e.g. Java) feasible?
Are there any specific pros/cons to using Julia in this type of application?
P.s. this is my first post, so please let me know if it would be more appropriate in another category. Thank you!
The front end will pass 10-15 doubles/floats to the engine. The engine will pass back 100-200 x/y coordinates as doubles/floats. The call latency isn’t as much of a concern to me as speeding up the thousands of Monte Carlo simulations. The alternative languages are Python or C#, which seem to perform those simulations much more slowly.
Compiling Julia is possible, and you end with a DLL, it’s not going to be small, but I question why you need that, and I think you will not get just one DLL with PackageCompiler.jl. Also it you think compiling to DLL or anything is needed for speed then that’s not true basically, and since this is a physics engine, it seem like soft-realtime, and that has special requirements, I’ll address lalter.
Certainly. Look at e.g. the PythonCall.jl project. It’s by-directional, i.e. ses it’s [py]juliacall.
But that and RCall.jl and such a project to call in the other direction, to call Julia, are not made for compiling the Julia code first to a DLL. Though I do not think it’s ruled out using those projects if you’ve already done that.
I would first as a prototype just try to use Julia from Java without thinking of a DLL, to see if it’s fast enough, and you like the language. Julia should be, can be, as fast as anything out there if you do things right.
See the discussion here/first post at least and my answer, in short, it’s not just for statistics or just from e.g. Stata:
I’ve not tried this, but it at least exists:
FYI: This is an older package:
You can call C from Java, and from C the C API, i.e. that way, or with projects helping, that in effect do this for you.
If it helps you and you can call C++ from Java (I’m sure you can) then it may be better to call Julia from it with, a project I’ve great confidence in/well documented:
The C API to Julia handles only basic types, one reason I point to the above C++ project, it relates to:
It’s also an option to call Python, and with the project above to Julia, in case it’s more mature then the other projects above.
The problem with mixing Java, a GC-language with another GC-language, such as Julia or Python, is that which GC should work (or both?). Java has good real-time GC available, Julia does not, so for a soft (or hard) real-time project you want to eliminate GC activity, meaning heap-allocations from the, or if soft-realtime, at least limit it. Though you may tolerate the Julia GC activity. In Julia instead of having a very good garbage collector for real-time, you rather want to eliminate the need for GC, and that’s a good approach, e.g. for robotics, but then you have to be careful. One project might help, it’s a bit limited yet, e.g. doesn’t support the GC, but for you it might be a plus: StaticCompiler.jl
Thank you! That’s a lot of helpful information that will take a while to fully digest. The reason for the DLL is that there’s an existing process that I have to follow, and using a DLL is part of it (my influence on the project is limited to the engine). Smaller DLL sizes and passing complex data types is important, so I’ll have to do some further digging on the links you sent. I appreciate the help.
I would echo the above comment. Although creating a Julia dll is technically feasible, you may be setting yourself up for a lot of additional technical work as you wade through the sharp ends.
Consider if a different remote procedure call (rpc) mechanism may be viable instead, especially since it sounds like doing the calculation may be much longer than communicating the actual data. For instance, you can set up your Julia process as a server and then make requests over http or by passing specialized structures between your Java and Julia code (see protobufs or flatbuffers and related techniques)
Julia can produce a DLL. As others have noted this is called a Julia system image and it can be built using PackageCompiler.jl. There is a complete example of this for an embedded application:
The problem for you is the interface. As the libcg example demonstrates you can use the @ccallable macro to create export a C function from the system image, which you can call. However, before you can invoke this function you also need to initialize Julia. The reality is that not alone do you need the system image, sys.dll, but you will also need libjulia.dll as well.
The Julia system image is not known for being particularly small. The other DLLs are not small either. If you thoroughly compile everything that you need ahead-of-time, you might be able to remove the compiler, libLLVM and friends, to make the total install size smaller.
Having Julia and Java within the same process can get a little messy because both languages actually operate in pretty similar ways. They both use signals for various purposes. One important mitigating factor is Java’s signal chaining facility: https://docs.oracle.com/javase/8/docs/technotes/guides/vm/signal-chaining.html
You also need to be very careful about multithreading and multitasking. The communication between Julia and Java should only be restricted to Julia’s root task and a single Java thread.
As others have suggested I would strongly consider a shared memory facility and some kind of interprocess communication mechanism to avoid these potential complications.
Overall, what you propose is possible, but there will be some challenges. If you do go down this road, make sure to ask a lot of questions here as you encounter issues.