Using Julia Compiled functions from Python with string arguments

General Usage
, ,
1

Hi
I am experimenting with Julia code that I compile with juliac. I want to use it from Python (from various reasons, one being that it is the language of choice for many colleagues). One problem I stumbled upon is that there is few examples of passing string arguments from Python to Julia and Julia to Python in the context of a library (working on MacOs 10.15, so Unix style).

One needs to provide a valid interface, compatible with C style strings. Naive implementation would be using Cstring variables, but it did not work when invoking function from Python. I ended up using Cwstring format. Which used to be less accessible, but I found this PR which did help a lot: No `unsafe_string` for `Cwstring` · Issue #28863 · JuliaLang/julia · GitHub

# Compilation as a Julia library using following command (in a makefile)
# time julia +1.12 ~/.julia/juliaup/julia-1.12.2+0.x64.apple.darwin14/share/julia/juliac/juliac.jl  --experimental --trim=unsafe-warn  --output-lib liblink.so --compile-ccallable ./link.jl

module Link

const version_st::String = "0.1"

Base.@ccallable function version()::Cwstring
    v = Base.cconvert(Cwstring, version_st)
    w = Base.unsafe_convert(Cwstring, v)
	return w::Cwstring
end 

"""
String conversion between Python and Julia through a compiled library with juliac requires on uses following types: Cwstring in Julia, ctypes.c_wchar_p in Python.
"""
Base.@ccallable function print_string(x::Cwstring, v::Bool)::Cint
    try
        v ? (println(Core.stdout, "Received string type(x): ", typeof(x))) : nothing
        z0 = unsafe_string(x)
        println(Core.stdout, "z: ",z0,", length: ",length(z0))
    catch
        v ? (println(Core.stdout, "Boom...")) : nothing
    else
        v ? (println(Core.stdout, "That worked fine!")) : nothing
    end
	return 0
end 

end

This compiles only with --trim=unsafe-warn, and throws a few (hard to read/interpret/exploit) errors.

Once this is done, I can turn to Python (which I am forced to learn by mere language-dominance). This code exploits the generated “.so” lib:

import ctypes
import pathlib

if __name__ == "__main__":
    # Load the shared library into ctypes
    libname = pathlib.Path().absolute() / "liblink.so"
    print("Loading library ...")
    print(libname)
    c_lib = ctypes.CDLL(libname)

    c_lib.version.restype = ctypes.c_wchar_p
    c_lib.version.argtypes = ()

    c_lib.print_string.restype = ctypes.c_int
    c_lib.print_string.argtypes = (ctypes.c_wchar_p,ctypes.c_bool)

    myst = "essai"
    res = c_lib.print_string(myst, False)
    print(f"Output print_string: {res}")

    res = c_lib.version()
    print(f"Output version: {res}")

Which gives me the following output:

MacBook-Pro-de-Me:protojl someone$ python3.9 test_modem.py 
Loading library ...
/Users/someone/Workspace/julia/protojl/liblink.so
z: essai, length: 5
Output print_string: 0
Output version: 0.1

In the end it worked, but not by any means straightforward. Hope that can help!

2 Likes
2

Why not just use JuliaCall or similar? That handles conversion of string arguments and other types for you. Forcing yourself to go through a low-level C API to communicate between two high-level languages is just going to make things painful.

3

Why not just use JuliaCall or similar?

I wish I could use JuliaCall/PythonCall. But I am behind firewall restrictions for work, and I cannot easily access Julia packages. I took this as a “figure de style” exercise.
I guess it is not the most usual case.

4

What exactly were your issues using Cstring?

I’m guessing encoding issues. You may have better luck changing the Julia function to take both the string pointer ::Cstring and the string length ::Cint (in codeunits). Python-side I would explicitly encode the string as utf8, like b=s.encode("utf8") then pass b for the pointer and len(b) for the length.

Also I don’t think c_bool and Bool are necessarily matching types. I’d use c_byte instead.

5

I did try to use ::Cstring instead of ::Cwstring, but I ended up getting only 1 character. Using string pointer Cstring and length Cint did not help (coredump), even when specifying the string length with unsafe_string.

Your suggestions seem useful, I will try it when I find time for that…

As for type correspondance, I looked up those pages: Calling C and Fortran Code · The Julia Language and ctypes — A foreign function library for Python — Python 3.14.2 documentation.