Calling Julia functions (which take custom structs as inputs) from Python

I’ve been trying to figure out a systematic way to call Julia functions from Python. I came across pyjulia library (in python) and PyCall modules (in julia) which seem to be pretty useful. However, most of the examples shown to demonstrate the usefulness of the packages, while extremely valuable and helpful, only consider inputs of type int, float, string, dict and arrays.
How can one call Julia functions that take other data types, say, custom Julia structs, as inputs? I’d really appreciate if somebody on this forum could offer some ideas/help on how I can call Julia functions with custom data structures other than the predefined types. I have provided an example below. The struct “Container” defined below is a simplified struct, and in a more general situation, could hold other custom structs. [I come from C++/Python background, and have heavily used Boost.Python and pybind11 to expose and extend Python to C++ and vice-versa. I am looking for something similar here]

#container.jl file:

export Container
export add_container_items
using LinearAlgebra

mutable struct Container{T<:Real}
x::T
y::T
function Container{T}(x::T, y::T) where {T}
return new(x, y)
end
function Container{T}(y::T) where {T}
return new(2, y)
end
end

function add_container_items(a::Container{T}) where{T}
y = a.x + a.y
return y
end

function add_numbers(x::Float64, y::Float64)
return x + y
end

Calling add_numbers() function from Python is straight forward, but how can I call add_container_items() function from Python, which takes a custom struct “container” as an input?

Thanks for any help in advance!

What happens if you call the constructor of Container from Python?

One solution, if you cannot construct a Container on the Python side, is to create a Python interface that accepts types that pycall can handle.

@baggepinnen, thanks for your suggestion.

I have tried:

from julia import Main

Main.TestModules.Container(4.0, 5.0). <----- calling container constructor
(I’ve placed this “container” struct under a module named TestModules)

However, this throws an error:
JULIA: MethodError: no method matching Main.TestModules.Container(::Float64, ::Float64)
Stacktrace:
[1] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:790
[2] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:789
[3] _pyjlwrap_call(::Type, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /Users/apg/.julia/packages/PyCall/BcTLp/src/callback.jl:28
[4] pyjlwrap_call(::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /Users/apg/.julia/packages/PyCall/BcTLp/src/callback.jl:49>

I think I kind of understand what you mean by creating a Python interface that accepts types that pycall can handle, but I am a bit lost on syntax (being a brand new Julia, PyCall and pyjulia user) I’d certainly appreciate any pointer.

Thanks a lot!

Have you imported the module on Python side (with PyJulia)?

In general, you can assign Python bindings to any Julia object and vice versa - in both languages, nearly everything is a first class object (incl. classes / structs, functions, etc.).

Consider the following Julia file “example.jl”

# example.jl
module ContainerTest

export Container
export add_container_items, add_numbers
using LinearAlgebra

mutable struct Container{T<:Real}
    x::T
    y::T
    function Container(x::T, y::T) where {T}
        return new{T}(x, y)
    end
    function Container(y::T) where {T}
        return new{T}(2, y)
    end
    function Container{T}(y::T) where {T}
        return new(3, y)
    end
end

function add_container_items(a::Container{T}) where{T}
    y = a.x + a.y
    return y
end

function add_numbers(x::Float64, y::Float64)
    return x + y
end

end # module

Take note, that I slightly changed the original definition of Container(y::T), because it is more usable, but for consistency added the original definition at the next step.

Now, in your python file, you can use this Julia file in the following way

from julia import Main
Main.include("example.jl")
Main.using(".ContainerTest")

add_numbers are straightforward:

Main.add_numbers(1.0, 2.0)
# 3.0

Simple container and add_container_items

c1 = Main.Container(1.0, 2.0)
Main.add_container_items(c1)
# 3.0

Container constructor with one argument

c2 = Main.Container(2)
Main.add_container_items(c2)
# 4

Container constructor with parametric type is more involved because there is no obvious python version, or at least I wasn’t able to find it. But you can use eval to build necessary variable

c3 = Main.eval("Container{Float64}(4.0)")
Main.add_container_items(c3)
# 7.0
2 Likes

@Skoffer, thank you so much for your detailed explanation ! That worked out nicely. I see the difference between your version and my version I was trying earlier. The template parameter “T” in the constructor was the reason why it was failing earlier. Moving that parameter to new{T} worked fine. As for the parametric constructor with {T}, looks like eval seems to be the way to go.

Now that this is at least working, I can try different scenarios where the Container member variables are other custom container/data structure other than predefined types like I had in the example above.

My only other question at this point is, how to properly manage the Main.include("") line in the code. For instance, in the Container example above (which is my custom library, say), let us assume that the custom struct and its functions make use of several core julia libraries/packages like Combinatorics, UniqueVectors, etc. to perform several mathematical operations for number crunching. The constructor and/or the function that I want to expose to Python will either take some custom structs (which I will expose to Python by calling the constructor) or other predefined types like array, numbers, dict, string, etc, which are already exposed to Python. They will never directly take any objects defined in core julia libraries. Inside the function body, which I want to expose to Python or call from Python, however, these functions could use one or many core Julia libraries to do the number crunching.

All of these other core julia packages are under my .julia/packages folder. Do I have to do Main.include(“path/to/core/julia/ package”) for each package used by the example above?

Is there a better way to accomplish this?

Currently, I am including these packages one-by-one. However, I keep getting errors due to the following lines:

Main.include( “/Users/apg/.julia/packages/RecipesBase/G4s6f/src/RecipesBase.jl” )

Main.include( “/Users/apg/.julia/packages/Polynomials/XpnvY/src/Polynomials.jl” )

Error:

ImportError: <PyCall.jlwrap (in a Julia function called from Python)

JULIA: LoadError: LoadError: ArgumentError: Package RecipesBase not found in current path:

Thanks a lot for your help! I really appreciate it.

Well, I think, that it can help if you try to run Julia code in Julia and only after everything compiles and runs correctly wrap it in python code. This way transition to the python environment should go much smoother since it adds a rather small wrapper around the usual Julia procedures.

I mean, you should treat your Julia scripts as usual Julia scripts, if you are using any libraries then they should be used in the usual way with using and other Module commands. You shouldn’t directly include library parts in python.

Consider the following example. In Julia’s example.jl you may write

using Roots
using Polynomials

function solve(a0, a1, a2, a3, x0)
    x = variable()
    poly = a0 + a1*x + a2*x^2 + a3*x^3
    find_zero(poly, x0)
end

Take the notion of using Roots and Polynomials at the beginning of the file. This is what is required by Julia’s code and Julia takes care of loading all the libraries properly.

In your python code you may use it as follows

from julia import Main
Main.include("example.jl")

Main.solve(1, 2, 3, 4, 0)
#  -0.605829586188268
2 Likes