I’m interested in the possibility of writing Julia functions as subroutines that can be invoked from a modern HPC code (i.e. something written in Fortran or C++, often with a compiler tuned to a specific supercomputer architecture, typically using a MPI+OpenMP model for parallelism).
What is the current status of Julia for this purpose? Is this possible, and if not, are there developments on the horizon? Can Julia interoperate with common parallel idioms in HPC codes, modern Fortran, and specialized compilers? Are there potential pitfalls with different memory management schemes? I know that Julia’s own parallel model is in a state of flux (and has been used for HPC applications like Celeste), but I’m more interested in whether it can interact with existing MPI/OpenMP codes.
It is certainly easier to call Fortran/C/C++ code from Julia than the other way around.
In an HPC context, I can envision a solution based on MPI (this is a very rough idea, I have not tested any of this, so please take with a bag of salt): There’s an MPI package for Julia and if one runs the HPC application in MPMD (multiple program, multiple data) mode, then some MPI ranks could be Julia processes while the other ranks would be Fortran/C++ processes, and then they communicate over MPI. But that would probably need some re-architecting of the HPC application. Also, actually launching the application in MPMD mode could be somewhat of a headache – in fact, I have never tried it…
Thanks. Re-architecting the HPC application is possible and probably necessary, but I’m hoping for approaches that are relatively non-intrusive.
In terms of the difficulties involved in calling Julia from Fortran/C++, do they have more to do with what has or hasn’t been implemented in Julia so far, or are they more fundamental obstacles involving the runtime etc. that would be shared by any dynamic language, like Python?
I don’t know much, but a couple links that may help while waiting for answers from those more knowledgeable:
Thread about embedding Julia into C++ program:
Compiling Julia into a shared library: