Julia to Python workspace transfer

I have some scripts in Julia and some in Python (unfortunately this could not have been avoided). It would be useful for me to save some of my Julia workspace variables (specifically, a dictionary consisting of keys and their corresponding Nx2 arrays) in a way that I can load the equivalent variables in Python. Is this possible with a package somewhere? I was thinking about the JSON package but was not sure this would work as I’d like to save a dictionary of arrays in Julia as a .json but was not sure how I would load this on the python side.

There are two different directions you could take here. The first approach, which using JSON would fall into, is to save your julia variables to disk then load them in Python. That’s certainly the simplest approach to implement. Python has good JSON support so it shouldn’t be a problem to read your arrays into Python. See e.g. this SO thread with tips on efficiently loading json into numpy arrays. If your arrays are large you might want to look at data storage format that more designed with storing bigger chunks of data in mind, such as HDF5. Again, julia and python both have good support for reading and writing HDF5 files. Use HDF5.jl on the julia side and H5py on the python side.

The above approaches are the way to go if you don’t care about the time it takes to save your arrays to disk, then load them back up again in the other language. If you need a lower latency solution then I would consider using PyJulia, which will allow you to call julia functions from python, or PyCall.jl, which will allow you to call python functions from julia. Both of these will allow you to share data in memory between the two languages. The catch is that it will be a lot more work to interface your julia and python codes this way than it would be to write the code to handle writing and reading the arrays to and from disk using JSON or HDF5.

I would write a Julia wrapper for Python scripts/functions using PyCall. Passing a dict of arrays in both directions should be pretty straightforward. If you need to save Julia workspace variables to disk, use JLD2. I had a similar workflow in a recent project.