so I am going to be populating a python data structure with financial instrument data. I want to use Julia to analyze the information. What is the most effective way to share the data between python and julia. thank you

[image] anon69491625: Let’s say that there is an existing and internally tested and ratified library of Python scripts. So they wish to consider using Julia to handle the parallel processing aspect of the next generation data visualization system. Wouldn’t it be more prudent to leave the existin…

PyCall? I don’t know if they’re able to literally share objects in memory, but this makes it quite simple to pass data back and forth. Is there some data structure that exists in python but does not in julia? Or is it just the library that does the work of building the data structure that you need? …

thank you for the reply, most kind. I don’t think Pycall will help me as I couldn’t see how it would share a Python data structure. SO to answer your question, I want to have the julia script access the Python memory directly. Here is a better description of the usecase. a python script sets up a…

[image] anon69491625: I don’t think Pycall will help me as I couldn’t see how it would share a Python data structure. PyCall can easily share data structures in memory with Python (libpython is loaded and running in the same process as libjulia).

It can??? could you point me at an example in the docs please? I haven’t really started using Julia yet but 2021 will be the make or break year. What I would appreciate is an example that does something simple. Just the Python script which creates a dataframe and the Julia code that accesses it.…

Every Python object (PyObject) in PyCall is shared in memory with Python. Python lists and arrays are converted by default when they are returned to Julia, but you can suppress the conversion by passing a specific return type (e.g. PyObject) to the pycall function or by accessing attributes as o."a…

hi there thank you so much for such a considered reply. I edited my reply to your previous before reading this one. What I was hoping for was a simple worked example as I don’t really want to know how the sausage is made :wink: BUT I am reading the docs tomorrow!! thank you again for your wonder…

[image] anon69491625: Just the Python script which creates a dataframe and the Julia code that accesses it. Here is Julia code that creates a pandas.DataFrame: julia> using PyCall julia> pd = pyimport("pandas") PyObject <module 'pandas' from '/Users/stevenj/.julia/conda/3/lib/python3.7/site…

I agree that I have been too lax in understanding the rudiments of julia. Isn’t my mental model a fair representation of what is actually happening? I create a dataframe in Python Magically I access the dataframe in Julia the dataframe in Python changes I see the same change in the Julia datafr…

[image] anon69491625: I create a dataframe in Python. Magically I access the dataframe in Julia. the dataframe in Python changes. I see the same change in the Julia dataframe. I think you need to distinguish between running separate Python and Julia processes and sharing data (much more di…

Let’s say that there is an existing and internally tested and ratified library of Python scripts. So they wish to consider using Julia to handle the parallel processing aspect of the next generation data visualization system. Wouldn’t it be more prudent to leave the existing code base in place an…

Sharing python memory

Internals & Design

stevengj December 26, 2020, 5:07pm 4

PyCall can easily share data structures in memory with Python (libpython is loaded and running in the same process as libjulia).

On Julia interoperability with C and PyCall

Topic		Replies	Views
Sharing a python dataframe in Julia General Usage	11	1451	January 27, 2020
PythonCall.jl and garbage collection General Usage	18	474	April 5, 2026
How to run an entire python script in pycall New to Julia pycall	41	7342	January 6, 2022
[ANN] PythonCall and JuliaCall Package Announcements interoperability , python , pythoncall	84	11351	May 8, 2023
How to copy the result of python to Julia without allocation memory using PyCall Performance package , pycall	8	508	December 12, 2023

Sharing python memory

Related topics