How to control Julia from an external non-Julia program

I want to to use Julia to create plots from data generated by an externally compiled program written in another language. I am on macOS Monterey using Julia 1.8.3. I want to control Julia from within the external program. Due to Julia’s time-to-plot issues and the large number of plots I expect to make, I would like to start a single Julia session and then send commands to it later, from the external program. The commands might be one-liners or stored in a file.

The language of the external program is not C but it has a binding to C’s System call by which I can send shell commands by simply appending a null character to the string of the command.

i have used this method to great success with a commercial program (Igor Pro) but I fear that Igor Pro is nearing end-of-life on macOS, thus my desire to change my workflow. The process is for the external program to write the plot data as double-precision floats to a file, then direct Igor via System to read and plot the data.

I have installed Plots and several of its backends as well as GLMakie so I am able to make plots from within Julia itself, including notebooks in VS Code and the VSC-Julia plot pane.

I have tried several things but am not progressing to a solution.

Igor Pro lets me access it by a command-line capability somehow built into its executable. This is fantastic. Can I do this with Julia? If so, how do I direct my commands to the “right” Julia since there sometimes seems to be several Julias running at once.

I suspect that I can achieve my goal by another method: by setting up a pipe to be read by the running Julia instance, then send Julia commands to the pipe from my external program using the System binding. Is this correct? How would I do this? (At this point, it is best to address me as an intelligent fifth-grader because I am not fluent in these things.)

A terrific bonus would be to be able to somehow send commands to Julia as it exists within a VS Code session, either a regular .jl file with its plotting pane or a Julia .ipynb notebook.

If you want to use a similar tool to what you are doing now you might look at DaemonMode.jl, this allows a persistent Julia session to be used to run scripts that are started independently.

Another approach would be to communicate with a Julia REPL through libexpect, which I believe is available on OS X though I don’t know for sure.

2 Likes

One option is to embed Julia:
https://docs.julialang.org/en/v1/manual/embedding/

If your external program can call C functions in a shared library, this approach can work. For example, I’ve embedded Julia into LabView.

Another approach would be to communicate with Julia over an IPC mechanism such as a ZeroMQ:

Yet another method would be to use IJulia which is used to embed Julia into a Juptyer notebook. I believe IJulia actually uses ZMQ:

You can also use Julia’s socket API to setup a TCP/IP server:
https://docs.julialang.org/en/v1/stdlib/Sockets/

If you want to move a lot a memory between an external program and Julia, you can memory map some shared memory:
https://docs.julialang.org/en/v1/stdlib/Mmap/

A structured way to that can be done via Arrow.jl which uses Apache Arrow:

1 Like

I think you’re on the right track with your pipe idea. I haven’t tried this approach with Julia, but it’s worked for me elsewhere: write your Julia program as if it were interacting with a user at the terminal, but use the Sockets package to communicate through a socket instead. Then your C program just needs to send commands over the socket to the port opened by the Julia program. You’ll distinguish among programs by their port numbers.