[ANN] PythonCall and JuliaCall

Package Announcements
, ,
58

PythonCall.jl is not under the JuliaPy organisation.

59

My bad. I’d assumed that the Julia orgs were similar to R task views: a convenient way to navigate the many packages available for a particular domain. Not so…

60

“Julia orgs” are simply GitHub organisations. There is nothing formal like those R task views.

61

EDIT: this message is about having plot() instead of plt.plot().

As a quick fix to run plotting code with PythonCall.jl, the following worked well. Do you know of a better/simpler way?

const mplot_funcs = (:figure, :gca, :subplot, :subplots_adjust,
                     :plot, :scatter, :bar,
                     :title, :xlabel, :ylabel, :xticks, :legend, :annotate,
                     :axhline, :axvline, :xlim, :ylim, :savefig )

for f in mplot_funcs
    @eval $f(args...; kws...) = plt.$f(args...; kws...)
end

and then the usual plotting

62

I’m just curious. How is it done? Python is known for its GIL

63

I could be misunderstanding, but do you mean just doing something like this?

So basically:

@py import matplotlib.pyplot as plt
plt.plot(<stuff>)

Edit: haha, also just saw @aplavin doing the same thing earlier in the thread!

64

Thanks for asking. The purpose of the message was to discuss how we could use plot() instead of plt.plot(). This would avoid rewriting code that currently uses PyPlot.jl.

65

Ahh, gotcha, thanks for the clarification! Would something like this work then?

@py import matplotlib.pyplot: plot, scatter, bar
1 Like
66

Yea I think that’s basically it. I made a tiny package to do this and also to make IJulia figures work right: https://github.com/marius311/PyPlotCall.jl I have no plans to develop this much further, but if someone wanted to claim ownership or build it out, please feel free to.

4 Likes
67

PythonCall used to have a very similar integration with IJulia which got lost in a code reorg. Would be great to reintroduce it.

2 Likes
68

Setting environment variables in PyCall is just one-off. However, for JULIA_PYTHONCALL_EXE in PythonCall, I need to set it to @PyCall or the Python path each time when I open the REPL. Is it normal?

69

That’s right, PythonCall intentionally doesn’t persist these settings. You can always set that env var in your .profile or startup.jl or etc…

1 Like
70

Thanks for the reply. Though I would prefer to have an argument for us to determine whether it memorizes the last setting.

71

Say I have a Julia function Main.f with two methods f(v::Array{Float64}) and f(v:Array{String}). With pyjulia (the Python module julia) I can do these calls: Main.f(['a', 'b']) and Main.f([1.0, 2.0]). In this case the Python lists will be converted to Array{String} and Array{Float64} and the correct method will be called. That’s the end of the story, I don’t need to do anything more. (The disadvantage is that I do need to do something more if I want to suppress automatic conversion.)

How can achieve the same thing with juliacall? For example Main.f([1.0, 2.0]) will convert the Python list to PythonCall.PyList{PythonCall.Py}, which is IIUC a wrapper around the Python object. And there is no matching method for f.

Because PyList <: AbstractVector, I could write a method f(v::AbstractVector). But, I would still need to look at the elements to find what the eltypes are and then convert and call f again. What would that code look like ? Or another option, perhaps better, would be to convert the Python list ['x', 'y'] to a Vector{String} from the Python side. This would be wrapped in the Python type juliacall.VectorValue. If I then call Main.f, I see that the correct method, one of f(v::Array{Float64}) or f(v:Array{String}) is indeed called. How can I do this; i.e. convert a Python list to a VectorValue ? I see that there is a function PythonCall.pytype. I could perhaps write some functions using these tools to do the conversions. Are there any facilities like this already written ?

EDIT: There is a function PythonCall.pyconvert(T, x) that converts a (python) object x to an object of Julia type T. I’d like to write a function pyconvert(x) that converts x to the most natural type. For example str → String. What is the most efficient way to write such a function?

72

The intended way to do this is

from juliacall import Main as jl, As as jlas
jl.f(jlas(["foo", "bar"], jl.Vector))

The object jlas(x, T) is viewed on the Julia side as pyconvert(T, x).

There was some discussion here with @icweaver about creating a function pyconvertnative(x) to convert only to Julia “native” types.

73

That doesn’t appear to work for me with juliacall 0.6.1
I get the following:

In [1]: from juliacall import Main as jl, As as jlas

In [2]: jl.seval('f(x) = nothing')
Out[2]: f (generic function with 1 method)

In [3]: jl.f(jlas(["foo", "bar"], jl.Vector))
---------------------------------------------------------------------------
JuliaError                                Traceback (most recent call last)
Input In [3], in <cell line: 1>()
----> 1 jl.f(jlas(["foo", "bar"], jl.Vector))

File ~/.julia/packages/PythonCall/Z6DIG/src/jlwrap/any.jl:167:30, in __call__(self, *args, **kwargs)
     28     module = Base
     29 elif mname == 'Core':
---> 30     module = Core
     31 elif mname == 'Main':
     32     module = Main

JuliaError: StackOverflowError:
Stacktrace:
     [1] pyconvert_rule_jlas(#unused#::Type{Vector}, x::PythonCall.Py)
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/juliacall.jl:52
     [2] (::PythonCall.var"#52#53"{Vector, typeof(PythonCall.pyconvert_rule_jlas)})(x::PythonCall.Py)
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/convert.jl:207
     [3] macro expansion
       @ ~/.julia/packages/PythonCall/Z6DIG/src/convert.jl:238 [inlined]
     [4] macro expansion
       @ ~/.julia/packages/PythonCall/Z6DIG/src/Py.jl:129 [inlined]
     [5] pytryconvert(#unused#::Type{Vector}, x::PythonCall.Py)
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/convert.jl:221
--- the last 5 lines are repeated 11424 more times ---
 [57126] pyconvert_rule_jlas(#unused#::Type{Vector}, x::PythonCall.Py)
       @ PythonCall ~/.julia/packages/PythonCall/Z6DIG/src/juliacall.jl:52
74

The following seems to work, and doesn’t seem grossly inefficient. Does this look reasonable? Am I am reproducing something that already exists, or is there a more idiomatic solution? (perhaps juliacall.As is convenient and efficient, once I get it working.) This converts list to Vector assuming the pyconvert succeeds for every element.

function pyconvert_list(::Type{T}, list) where T
    vec = Vector{T}(undef, length(list))
    for i in eachindex(list)
        vec[i] = PythonCall.pyconvert(T, list[i])
    end
    return vec
end
75

That’s a perfectly reasonable function, but I don’t think it’s any different from pyconvert(Vector{T}, list)?

Thanks for the bug report, there was a very silly mistake in the conversion code. Fixed on main. I think I’ll make a release today too, so try again in an hour.

1 Like
76

pyconvert(Vector{T}, list)

I didn’t try that. I think this is exactly what I’m looking for. A minor problem is, from the python side, jl.Vector works, but jl.Vector{String} is not valid python syntax. I suppose you could do jl.seval('Vector{String}').

77

You can do jl.Vector[jl.String] instead. :slight_smile:

I really should work on the JuliaCall docs, they aren’t great!