Is PyJulia org, e.g. juliacall (or pyjulia) Python packages maintained?

New to Julia
1

Hello, I’m just checking if PyJulia is maintained or if there is a better way to call a Julia script from a Python code.

2

Only minimally — juliacall · PyPI is more up to date and actively developed.

9 Likes
3

Thank you for the response.

Is it possible to call a separate Julia script from Python using Juliacall? I was able to do using PyJulia

4

@MilesCranmer recently picked up maintenance and released pyjulia v0.6.2 a month ago:

He previously asked a similar question as this.

Regarding juliacall, see the documentation here:

7 Likes
5

Thank you!

6

I’m not sure what you mean by “call a script”. You can call functions from Julia libraries (packages and modules), and you can run scripts (.jl files) and load their definitions (by running include).

1 Like
7

FWIW I would say that PyJulia is still unmaintained; I wouldn’t want to give the impression that my tiny PRs are anything more than keeping the lights on. juliacall is still highly experimental and has issues with standard Julia features like multithreading (Bus error with multithreading linear algebra · Issue #298 · JuliaPy/PythonCall.jl · GitHub).

I also want to take this thread to bump my comment here:

and

I want to argue that those in charge of Julia funding need to direct some to a dedicated team working on Python interop, rather than relying on community maintenance. I really do not see any other part of Julia development as important as this, with such a large a return on investment.

21 Likes
8

A lot of effort is currently being devoted to make Julia easier to deploy, this entails making it easier to ship compiled binaries, both statically compiled and binaries including more of the run time. This work benefits not only Python-Julia interop, but interop with potentially all other programming languages that are capable of calling C code.

5 Likes
9

@baggepinnen thanks for this. Could you please share a bit more about this and how it would benefit Python interop? More generally, where can I read about these efforts? I have not seen any ongoing work on this front in the PythonCall repository, for example.

10

It’s not directly related to python at all, it just happens to be useful for people who want to call Julia code from python.

The discussions and work around this are somewhat spread out, see, e.g.,

3 Likes
11

Yes PyJulia unmaintained, but juliacall is, and I believe it experimental is misleading, simply untrue. PythonCall, and juliacall, that is based on it (a little bit of extra code, PyJulia is a much larger package, that that extra code, thus harder to maintain), are not “experimental” (I find one exception in the manual for the juliacall IPython extension: “The extension is experimental and unstable - the API can change at any time.”)

Julia does support multi-threading yes, but not by default (except for BLAS, which I think just works with, is not the issue below), you have to start it a certain way for it, so in some sense it’s NOT a standard feature, and I think never will be enabled by default.

I believe juliacall may be better on all counts already (though startup still a bit slower), why e.g. diffeqpy migrated from PyJulia to juliacall. Only thing is the former has better conda integration, but it’s worked on and seems close, and not needed for all. [juliacall had an issue with Apples M1 macs, now fixed, I’m not sure, very possible PyJulia has a similar issue, or not, but the fix was simple.]

Does PyJulia actually support multi-threading (or better than below)? It’s not fully clear to me, then an advantage, but I see in the, yes, still open issue you posted (was just scanning it): Bus error with multithreading linear algebra · Issue #298 · JuliaPy/PythonCall.jl · GitHub

:star: Happy Update :star: : After several weeks of troubleshooting, I think we have a robust and minimally invasive solution that doesn’t require switching away from juliacall/PythonCall towards something like PyJulia.
[… code]
Disclaimer: This has only been tested on Julia 1.9.1+ and Python 3.8. Some threading logic has been updated in the julia runtime and released as part of Julia 1.9.1, see discussion here, that is likely allowing this to work.

The code there uses @threads i.e. is not just normal linear algebra/BLAS-using code, and it will work on defaults, when Julia isn’t multi-threaded, or with the workaround shown there, seemingly, if threading enabled.

12

If creating a Python interop package becomes easier in the future because of easier static compiler tools, that’s great! But we still need someone to actually set it up.

Rust is currently getting adopted by large parts of tech industry. This is only possible because of good C++ interop. Why shouldn’t Julia be the same for Python? I think a lot of this is due to the Python-Julia interfaces not getting enough support, which would otherwise make it significantly easier to start gradually replacing parts of large legacy codebases with Julia.

Good Python-Julia interop should be a top priority for Julia.

4 Likes
13

To be clear, I’m not saying JuliaCall is experimental while PyJulia is not. They are both experimental! And that is bad. Having just tried to set up JuliaCall support for one of my libraries, I can say with firsthand experience that it is certainly in a development phase (as is PyJulia!). At the end of the day these packages do not receive ample support from Julia considering their potential impact.

14

The julia developer survey explicitly asks users what they like and dislike about Julia. Interop with other languages was considered the least problematic technical feature, indicating that this is perhaps not as big of a concern for most people as you might think it is? Only 3% of the respondents considered this a problematic area. Granted, people are blocked by this are perhaps not using julia and are thus less likely to respond to this survey, but I still think it’s an indication that there are other, more pressing concerns to work on.

3 Likes
15

Yeah… This is classic survivorship bias like the plane meme that gets posted everywhere –

image
image1200×894 103 KB

Basically, you should be careful when looking at a survey of users who adopted something for answers about expanding adoption, as their answers are basically useless in that context.

What we really need is a survey of non-Julia users for why they aren’t using Julia :slight_smile:

22 Likes
16

I wasn’t trying to discount your “firsthand experience”. I thought you meant it was (officially) claimed experimental.

Technically interop can be about calling other languages (maybe interpreted that way), or calling from other to Julia, and PyJulia or similar packages could be in worse shape, so possibly some Python survey relevant… or not.

Also, people could be interpreting as calling C or Rust, or from , both possibly excellent. I’ve used PyCall at the time without problem, but PyJulia builds on it, with lots of code, and I only minimally tried it.

What are your issues with juliacall (or PythonCall.jl) other than relating to multhreading (which can always be avoided by at least not enabling it)?

[The missing conda support isn’t like a bug, just a partially missing feature, and coming soon it seems. Note Conda/Mamba is actually used for dependencies, one pro over PyCall.jl.]

17

I would expect that the majority of Python users do not want to “call Julia in python,” rather they would want Python interfaces to Julia libraries.

One example of this being apparently challenging to pull off is the duplicate implementations of Clarabel.jl and Clarabel.rs libraries — both individually excellent, but the python clarabel interface chooses to call the Rust library. maybe the authors have insight as to more specific challenges in calling Julia from Python

2 Likes
18

And that’s actually possible, then you need a Python package depending on juliacall. It will download Julia for you.

Some people, understandably (though not all), will dislike the heavy dependency Julia, but it DOES work, it’s just a large (automatic with juliacall, unlike PyJulia) download.

[When you can make small binaries (you can in some bases, also libraries), with e.g. StaticTools/StaticCompiler, the issue goes away", but yes, it doesn’t support all Julia code. And this is still an issue people have with Julia, the survey confirms, and it CAN be related to interop. I.e. strictly there’s no problem call Julia for from Python, just do you want to. This is only a “problem” regarding calling Julia, when calling from you already have the large Julia, it can be a problem for distributing Julia programs, but that’s solved with AppBundler.jl if you want that. It bundles all your dependencies, at least Julia packages, also PythonCall.jl, but I’m not sure if it bundles Python dependencies too, or if they are just auto-downloaded in first use.]

19

This supposes that Julia is centrally funded, but it is not. Julia funding comes from a variety of sources.

Officially, the Julia Language open source project has its accounts fiscally managed by NumFOCUS. The ledger for the project is available here:

If someone committed to organizing JuliaPy into a project, it could probably apply to NumFOCUS as an independent project as JuMP and SciML already are. That would probably make sense in that JuliaPy serves to connect NumFOCUS projects.

4 Likes
20

(For reference I was more referring to JuliaHub)