[ANN] PythonCall and JuliaCall

Package Announcements
, ,
45

Can you elaborate on this? pytable sends a Julia table to Python, PyTable wraps a Python table as a Julia table, so they are opposites.

46

Thanks for all the great questions everyone. I shall collate them into an FAQ.

4 Likes
47

Maybe thinking like multiprocessing - so having one python.exe for each thread

48

Coincidentally, we talked about this on the Julia HPC call yesterday (@jblaschke @staticfloat). The thing with one interpreter per thread is that you get in the “distributed business” of having to do something like @everywhere pyimport("numpy") and so on.

49

Do you mean subinterpreters (PEP 554)? It looks like that it is still work-in-progress. It’s definitely an interesting addition to Python. But one of the concerns was that it complicates C API usages. Also, since it looks like that at least the first version of subinterpreters share the GIL, we need to release GIL at Python-Julia boundary anyway (I’ve been wanting this in PyCall) to make it useful.

1 Like
50

I’ve tried using PythonCall + CondaPkg from pluto notebooks. Everything basically works fine: packages get installed and imported, plots displayed:

Screenshot

image
image597×793 46.7 KB

The packages themselves seem to exist in a single copy in ~/.julia/scratchspaces even if I create multiple notebooks that use matplotlib - they don’t duplicate for each notebook. I don’t really understand how this works though…

Still, new directories in /tmp are created for each new notebook run. They are created by Pluto, but Pluto only puts Project & Manifest files there that don’t take any noticeable space. “Apparent sizes” of those temporary dirs (speicifcally, the .CondaPkg subdirs) are 1 Gb, but actually most of the files are hardlinks I assume: each dir adds 150 Mb to disk usage, not 1 Gb. Is it possible to remove this duplication?

1 Like
51

thank you for the reply, I need to re-word to elaborate on what I meant.

having separate methods for conversion is fine, however it would be nice
to have separate method names entirely rather than different casing, to
minimise potential confusion.

alternatively there could be a flag/arg to specify conversion direction?

53

Good to know not everything is copied. I doubt there’s anything we can do from Julia to reduce this, it’s just how Conda works I think.

54

It’s a naming convention you’ll have to just learn I’m afraid.

The good news is it’s consistent: pyint, pylist, pydict, pytable etc all convert a Julia object to a Python one, and PyList, PyDict, PyTable etc all wrap a Python object to give a more Julian interface.

The reason the latter are capitalised is that they are classes: e.g. PyList{T} <: AbstractArray{T}. The former are lowercase because they are all functions and all return Py.

1 Like
55

PythonCall should work fine with Julia’s multiprocessing packages.

Having multiple interpreters or threads within a single process isn’t currently supported.

56

Thank you for the detailed explanation!

57

Noticed that the JuliaPy github organization did not list PythonCall among the repos – it should.

58

PythonCall.jl is not under the JuliaPy organisation.

59

My bad. I’d assumed that the Julia orgs were similar to R task views: a convenient way to navigate the many packages available for a particular domain. Not so…

60

“Julia orgs” are simply GitHub organisations. There is nothing formal like those R task views.

61

EDIT: this message is about having plot() instead of plt.plot().

As a quick fix to run plotting code with PythonCall.jl, the following worked well. Do you know of a better/simpler way?

const mplot_funcs = (:figure, :gca, :subplot, :subplots_adjust,
                     :plot, :scatter, :bar,
                     :title, :xlabel, :ylabel, :xticks, :legend, :annotate,
                     :axhline, :axvline, :xlim, :ylim, :savefig )

for f in mplot_funcs
    @eval $f(args...; kws...) = plt.$f(args...; kws...)
end

and then the usual plotting

62

I’m just curious. How is it done? Python is known for its GIL

63

I could be misunderstanding, but do you mean just doing something like this?

So basically:

@py import matplotlib.pyplot as plt
plt.plot(<stuff>)

Edit: haha, also just saw @aplavin doing the same thing earlier in the thread!

64

Thanks for asking. The purpose of the message was to discuss how we could use plot() instead of plt.plot(). This would avoid rewriting code that currently uses PyPlot.jl.

65

Ahh, gotcha, thanks for the clarification! Would something like this work then?

@py import matplotlib.pyplot: plot, scatter, bar
1 Like