Call compiled Julia function from Python

New to Julia
1

Hello, I have a Julia function that accepts a sparse matrix and some parameters, and returns two sparse matrices as a result.

I know that there is a Pycall, but I don’t want to call an active instance of installed Julia, I want to call a compiled extension similar to how it is possible to have C extensions for Python, so that there is no need to install Julia to use this function.

I understand that I should probably use PackageCompiler, but I don’t understand whether I should create an app or a library. Is there an example of this working end to end?

So if I have a function, do I need to convert it to a package/module? When I get compiled artifacts, how do I use them inside Python? Will I be able to pass a sparse numpy matrix as input and get it as an output? There are a lot of unclear points and I would be grateful if someone could guide me.

3 Likes
2

This is what you want:
https://julialang.github.io/PackageCompiler.jl/stable/libs.html#libraries

You want to create a C-callable function that will be compiled into a shared library. Once you have that, you can load the library into python and call the function just as you would for any C shared library.

1 Like
3

Do you have a toy example of how this would work?

Also how do I combine Julia sparse matrices, numpy sparse matrices and C?

4

Why don’t you want to use PyCall? Or in the case pyjulia (a related package). PyCall works very well, for the calling part. The reason to avoid it, is having to install Python packages, using two package managers, and Python’s just not as good as Julia’s. Those arguments against using together do not seem to apply to pyjulia.

Whatever you do, I would at least prototype first using pyjulia, and see if that’s just not enough. I haven’t looked at the latest option for producing libraries, in PackageCompiler.jl but one issue before was that static apps with it are rather huge, since it actually bundles Julia with the app. I suppose that’s the case with libraries too.

5

In the docs, there is a link to here:

6

Because I want to create a fast module for Python. It is not feasible to ask user to install Julia to use your package. So I want to get a binary or C version of what my function does in Julia, so I can call it from Python.

7

Yes, I saw this repo, but this is neither toyish nor end-to-end. I was hoping that there is a tutorial of some kind.

8

Creating a C-callable function and using PkgCompiler might be the best option. One advantage of supplying a dynamically linkable library is that people are used them; it won’t scare them away. But, it’s not a lean library. It still contains the Julia runtime (although in recent development versions of Julia you can get rid of the compiler, if I’m not mistaken)

There is also the python module juliacall. You can make this a python dependency that you can pip-install. It will automatically download and install Julia if it’s not detected in the user’s path. You could perhaps use the jill python library to roll your own installer for your project and use either juliacall or pyjulia.

2 Likes