Indeed it’s perfectly valid to do that. My next focus is going to be on getting the documentation and testing better. BTW The JULIA_PYTHONCALL_PROJECT variable might go, and instead we’ll just respect JULIA_PROJECT instead if it is set.
You can also specify the output type like @pyeval x => "1+x" => Float64.
I’ve encountered that issue before in pyjulia but don’t actually know its cause.
I imagine the difference is in how the packages load libpython. In JuliaCall, we pass ctypes.pythonapi._handle to PythonCall, which is a pointer to an already-open libpython. I assume PyJulia/PyCall opens libpython itself.
An older version of PythonCall hooked into IJulia so that matplotlib plots were automatically shown, just like they do in IPython. I could certainly add this back.
Yes indeed, I’ve been thinking that CondaPkg should respect the standard Julia environment variables better. I’ll change it to use JULIA_PROJECT if set.
A technical question, if I may: A current issue with pyjulia is that there’s trouble if Phython is statically linked. That doesn’t seem to be the case with juliacall, how do you get around that issue?
Awesome work. However, I encountered the following error in Julia 1.8.0-DEV when trying the example above and Julia exits immediately:
julia> using PythonCall
julia> plt = pyimport("matplotlib.pyplot")
Python module: <module 'matplotlib.pyplot' from 'D:\\DeepBook\\.julia\\environments\\v1.8\\.CondaPkg\\env\\lib\\site-packages\\matplotlib\\pyplot.py'>
julia> plt.plot(randn(100))
qt.qpa.plugin: Could not find the Qt platform plugin "windows" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Thanks for your efforts on PythonCall and JuliaCall, BTW. When I created PyCall eight years ago, it was an urgent need to compensate for the relative paucity of Julia packages, but some types of interoperability were still difficult due to Julia limitations. For example, I spent years bugging @jeff.bezanson about dot overloading every time I ran into him, which we finally got 4 years later. Zero-based arrays were not well supported in Julia, and in general it was tricky to decide which Python types should be converted to native Julia types (for convenience) vs wrapped (for efficiency & flexibility) — in hindsight, with the Julia features we have now, I would have left more Python types as-is except for types with lossless round-trip conversions. The very first version of pyjulia was written by Fernando Perez as an “IPython magic”, but it was years before anyone would want to think about calling Julia from Python as more than a cool demo. And, while PyCall could originally link to any libpython at runtime, it had to be switched to build-time configuration in order to improve load times; fortunately, Julia’s loading speed has improved since then.
Nowadays, the need to call Python code from Julia is less urgent, though I still am in the habit of using Matplotlib, while the desire to call Julia code from Python in “real” code is growing, and it seems reasonable to re-think many of the design choices that originally went into PyCall. Meanwhile, I haven’t had as much time for PyCall development effort, and while we’ve toyed with the idea of a “PyCall 2.0”, it would be disruptive enough that it’s about as easy to switch to a completely new package. So I’m quite happy to see people actively working on an alternative re-design, and hope that the existing PyCall and PyPlot are helpful in this process.
Note that this was deprecated some time ago in PyCall, and I wouldn’t recommend bringing it back in PythonCall. It doesn’t make the code any shorter or easier to write, and it obscures what is actually going on. The goal is to interoperate with Python code, not necessarily to mimic Python syntax in Julia.
Also, as a general principle, one should be cautious in programming languages/libraries about having multiple syntaxes for the exact same thing, since it can just add confusion and complexity for little benefit.
You can do @py import numpy as np (note the space before import).
I’ll write about it more another time but @py is an experimental macro which maps Julia syntax to its Python equivalent - not just import but many other operations too. You can write big blocks of code with it. Despite @stevengj’s concerns above, which are perfectly correct, this macro can sometimes simplify syntax considerably. It can also make tight Python loops much faster by avoiding Julia’s GC.
If I’m honest, I have partly put off writing this announcement because my intention was never to write “PyCall 2.0” - it started out as an experiment to see if I could do some things differently and just snowballed. I’m extremely pleased and relieved you have such a positive reaction.
Indeed PyCall is a long-established package and I can totally see where some of its quirks come from, historically. The above comparison was never intended as an attack on PyCall - quite the opposite, it’s a brilliant package and (not least) is responsible for some very handy syntax that PythonCall can take full advantage of without the baggage of backwards incompatibility.
I have thought about merging some of the ideas from PythonCall into PyCall - the world doesn’t really need two Python-Julia interop packages - but some of the changes would be so dramatic that, like you say, a totally new package would be no less disruptive.
There should be more messages from JuliaPkg explaining why, namely it couldn’t find a compatible version of Julia to use. So either Julia isn’t in your PATH, or it isn’t new enough to work with JuliaPkg which requires Julia 1.6 or above.
Also, aside from the Julia installation itself, it won’t take up any more room on your disk because it will use your usual Julia depot.
I think I’ll wait for someone else to reproduce before opening an issue. I have multiple installations of Python in different directories and some of them are added to the path and the same thing goes for Julia. I feel that this could confuse the proper installation of PythonCall, so I’ll investigate more and try on another computer before I open an issue. Many thanks.
Edit:
Well, I got it fixed now. The installation was somehow corrupted, I removed the package, then removed everything in the CondaPkg folder here: .julia\environments\v1.8\.CondaPkg and reinstalled PythonCall. When I issued plt = pyimport("matplotlib.pyplot"), the package complained about matplotlib module not found. I simply did pip install matplotlib in Windows terminal inside .julia\environments\v1.8\.CondaPkg\env\Scripts and everything worked like a charm. Thank you for the great effort and awesome package.
Nope, no messages and I have Julia1.8DEV in the path. Well, don’t know if it makes a difference but what I have in the path is C:\programs\julia-1.8 where a julia.bat takes charge of calling the julia.exe
Good to hear that.
BTW, a better explanation/example on how to call Julia packages from Python would be nice. As is, sorry but I just gave up.
Is there a reason not have PyCall 2.0 simply be PythonCall and deprecate the PythonCall name? I haven’t thought about all the implications and whether it even really makes sense for any real-world use cases, but then packages that only use the common subset of features could have a compat entry allowing both “classic” PyCall as well as PythonCall.
Yes, some reasons. PyCall 2.0 will by definition be a breaking change, so the name might as well be PythonCall. It IS possible to use PythonCall with the current PyCall (it’s a non-default option), so that’s one argument for the new name.
At least there’s no way to use some version of a package, such as PyCall, directly or as a dependency, and also another version, through another dependency. I’ve previously thought it might be useful, even for minor versions, just because some dependency is holding back, but I think the right thing is to upgrade to a new (minor) version (and make, trivial usually,PR to packages, as I’ve done, to make it possible to use there latest versions for all packages).
Do any other language, or their package managers allow this?
A good (non-technical) reason is to give @cjdoris (full) credit for his own package, which is a rewrite, not largely (almost not at all?) built on PyCall. He deserves it. PyCall is also really good, and I’m not sure @stevengj would want it taken over (in this way, something similar has happened, I believe, at least once, with Graph.jl). I would advocate he links to PythonCall as an alternative.
One difficulty is that it’s not just different features, but the semantics are quite different. And the scope, in terms of aspects of a project that are supported, is quite different. I don’t think just a few compat functions will be sufficient to make PythonCall a drop-in replacements. (I’m not sure if that is what you are suggesting.)