Merge Python and Julia, i.e. an (unofficial) integration proposal, sort of like the Mojo language

@cjdoris has an excellent package PythonCall.jl, and it means you can call to or from Python. I just don’t think that most Python users know of it (or the alternative pyjulia/PyCall.jl).

I know we have excellent integration already, and maybe most Julia users also know it, but the proposal is to draw Python users in.

Julia solved the “two language problem”, at least to a degree Python doesn’t, but those users mostly don’t know or care it’s an issue, and while this proposal seems to abandon it, I think I can justify it. It might seem heretical to suggest this, like giving up, but I’m thinking of the Python users mostly (and how it could selfishly benefit the Julia community to draw them in).

What I like about Julia is e.g. the awesome REPL, and the Python REPL is just inferior (also the default Ruby REPL, let alone Perl one and in any other mainstream language). [Does Python have some good opt-in REPL, similar to non-default Ruby REPL?]

Phase 1:

I’m thinking Python users, could start python, the same default python binary executable, as quickly as before, and they would not bear any interaction cost, until or if they they interop with Julia.

Same for Julia if you run the julia binary. It’s only that this unofficial self-extracting bundle.exe would install both languages/binaries. The combined download size is of course larger, roughly sum of both, but not a big issue, a one-time cost for all users (until we think of compiled Python and/or Julia programs).

Phase 1 seems like really easy to implement, possibly useful to (some) Julia users, but premature to announce to Python users until at least Phase 2:

Phase 2:

The Julia REPL is too good for Python users to pass up on. Only having it additionally for the python project seems doable, you could edit python code, it would be less of a drag, and you would only pay the slower Julia startup cost for an interactive section of Python.

This seems like an easy additional step, also what I would prefer, in 2b, a keystroke to go back and forth from Python to Julia. And from either language mode of course ; to enter the shell.

Phase 3:

We’ve had JuliaPro before, with a curated list of Julia packages. Here things get interesting. I think we should include Make.jl plotting package, and it would really interest interactive data science Python users. They could of course install their matplotlib as they’re used to, and use whatever plotting package from either language, but it seems warranted to include Makie.jl in the download. It might not need to be included into a default environment. I’m not sure we should have anything preinstalled, that way. At least we need to allow upgrading independently, unlike is done for Julia’s stdlibs. I’m open to a list of packages preinstalled in the package environment, that people could of course just delete from it.

This could go both ways, if there are some excellent Python packages Julia users should be aware of or might overlook, that could be added and usable from the Julia side.

@ChrisRackauckas’ SciML/DifferentialEquations.jl seems possibly very specific, too specific for most users(?), but I’m not opposed to including it in some way, at least diffeqpy (something I think is not too large to bloat the download, and I assume it autodownoads the rest on use):

Polyglot Userbase
While the majority of the tooling for SciML is built using the Julia programming language, SciML is committed to ensure that these methodologies can be used throughout the greater scientific community. Tools like diffeqpy and diffeqr bridge the DifferentialEquations.jl solvers to Python and R respectively

What I’m not proposing is also integrating the R language… I mean we could go all in, and RCall.jl already has an excellent R Mode, that I want emulated for Python. It’s just that Julia 1.10 is finally GPL-free, and R is GPL, and I want to keep Julia, and this project, GPL-free. Possibly the package RCall.jl only should be included though… RCall.jl doesn’t install R for you but it uses R that is preinstalled. That’s unlike PythonCall.jl that downloads Python for you or Julia if calling from the Julia side. Which would means no longer needed, to have an internet connection, after first install.

It seems obvious we would want to use the latest stable Python version (currently 3.11), but 3.12 will have a release candidate by the end of this month, and should be released 2023-10-02, so it’s tempting to use that also since:

The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.

Python plans to adopt UTF-8 fully (internally) and that might be it (or just part of the plan?). Currently passing arrays (of Float64) between Julia and Python is no-copying, i.e. fast, but passing strings work, while I think it means on O(n) operation. With both using UTF-8 internally it should be O(1).

On the Julia side again latest version (currently 1.9), or maybe latest LTS (will likely be 1.10, now in alpha2?).

Mojo promises very fast “Python” but really it’s also two languages in one, the new one resampling Python syntax, new semantics. I’m really proposing similar, it’s just two in one with different syntax and different semantics.

It will e.g. be confusing to Python users to use 1-based by default, but most wouldn’t write their own Julia code, at least at first. You could add OffsetArays to the mix, installed by default if it help to have 0-based. I expect most Python users would be tempted by just calling state-of-the-art Julia code, and get them interested in Julia potentially.

I think we need to preempt Mojo getting larger mindshare than Julia, since we are ahead in many ways (not all) already. Mojo and Julia have many similarities, like speed.

Chris Lattner, Mojo and LLVM guy on Julia:

What I like about Julia is e.g. the awesome REPL, and the Python REPL is just inferior (also the default Ruby REPL, let alone Perl one and in any other mainstream language). [Does Python have some good opt-in REPL, similar to non-default Ruby REPL?]

IPython is closer in capabilities to the Julia REPL. It is nice that the Julia REPL is good by default, though.


First, I like the idea of heading Mojo off at the pass :stuck_out_tongue_winking_eye:

I have some questions:

What workflow are you imagining? I’m writing a project in Python and then how does this change it? Can I throw in a import Makie at the top of my file and the hybrid language interpreter will pull in Makie with PythonCall?

Or if I’m working on the REPL and define const str = "Hello World" on the Julia side, what happens to str on the Python side? Are the Julia values all isolated under the Main module? If so, how is this any different from what PythonCall already is?

As it stands, this proposal seems like it’s just PythonCall + a Python REPL mode. What am I missing?