Calling julia functions from python

General Usage
,
1

I’m interested in using Julia to speed up parts of a python program I’m writing.

It seems pyjulia GitHub - JuliaPy/pyjulia: python interface to julia is the standard package to do this with? As it uses pycall I guess it should be fine to call the function with some arguments being ndarrays (from numpy)?
Looking at pyjulia, calling stdlib functions and evaluating julia expressions seems pretty straightforward. But I don’t understand how one is supposed to, let’s say, import a julia function from a .jl file and then call it with some (python) arguments?

4 Likes
2

I think pyjulia needs a little love. I distinctly remember figuring this out a long time ago, but am unable to do so in an easy fashion right now.

The best I could come up with is to take advantage of the fact that, when you include a file in julia, the last “value” in the file is returned. If the last thing in the file, is a function, then you get back an anonymous version of that function:

# test.jl

function myArrayFn{T<:Number}(x::Array{T})
    println("array size: $(size(x))");
    println("max element: $(maximum(x))")
    println("min element: $(minimum(x))")
    return 2x
end

In [1]: import julia

In [2]: j = julia.Julia()

In [3]: fn = j.include('test.jl')

In [4]: fn
Out[4]: <PyCall.jlwrap myArrayFn>

In [5]: import numpy as np

In [6]: x = np.array([[1,2,3], [4,5,6]], dtype=np.float64)

In [7]: fn(x)
array size: (2,3)
max element: 6.0
min element: 1.0
Out[7]:
array([[  2.,   4.,   6.],
       [  8.,  10.,  12.]])

This only works for the last function in the file.

I tried a number of other things (using include, include_string, etc.), and nothing seemed to modify the Julia environment available in the python Julia object. It might be that the internals have changed, and that pyjulia hasn’t kept up. Or maybe I’m just doing something wrong. I haven’t tried digging into the code to see why.

Anyway, hope this helps, and maybe someone else will respond with better info.

Cheers,
Kevin

2 Likes
3

Figured it out. With foo being an exported function in test.jl:

import julia
j = julia.Julia()
j.include("test.jl")
j.eval("importall julia_util")  # I used importall, sure other stuff also works.

Now you can define your function with foo = jl.eval("foo").

Or you do j.add_module_functions("julia_util") and then use j.foo.

6 Likes
4

This is explaining how to use Python on Power BI, my question is, how can I use this package in Python at Power BI to call Julia file, where shall I save the julia one?

1 Like
5

Greetings,

Thank you for providing a simple example to show how to call a function in a Julia file from Python.

But this example only works for one function in a Julia file. How about there are multiple functions in the Julia file? I just put two simple functions, i.e. testFunc01, testFunc02 in the .jl file. When I call it as shown in the example, it said ‘Runtime Error: Julia exception: MethodError: no method matching testFunc02(::Int64)’.

Can anyone tell me how to handle it and call any one of the multiple functions?

Thanks,

Larry

6

You can do something like

from julia import Main
Main.include("script.jl")
Main.testFunc01(x, y)
Main.testFunc02(a, b, c)

It looks like you are passing an unexpected argument to testFunc02. You’d need to provide an MWE to get more specific advice.

Also, please read Please read: make it easier to help you

6 Likes
7

@tkf It worked perfectly! Thank you so much! I hope this post can help more others who have the same question.

8

First, @tkf thank you very much for your suggestion!

I am developing a small simulation project (a dozen of files), currently completely in Python (numpy, scipy). I would like to port the project parts which perform the actual number crunching to Julia, and leave the rest of the project like input parsing and output presentation as it is just working (and as I’ve zero experience in Julia). I would like to do the port file by file.

Now, I need to import several Julia files/modules into a Python file, with a separate name space for each Julia file. Like following:

Python project:
PurePyMain.py

import pyscr1 as scr1
import pyscr2 as scr2

print(scr1.testf_1(2), scr2.testf_2(1))

pyscr1.py

def testf_1(x):
  return x*0.1

pyscr2.py

def testf_2(x):
  return x*0.2

Julia-in-Python project (what I want to have) :
PyJlMain.py

import jlscr1 as scr1
import jlscr2 as scr2

print(scr1.testf_1(2), scr2.testf_2(1))

Julia-in-Python project (what I got to work) :
JulPyMain.py

from pyjulint1 import Main as scr1
from pyjulint2 import Main as scr2

print(scr1.testf_1(2), scr2.testf_2(1))

julscr1.jl

function testf_1(x)
  return x*0.1
end

julscr2.jl

function testf_1(x)
  return x*0.2
end

pyjulint1.py

from julia import Main
Main.include("julscr1.jl")

pyjulint2.py

from julia import Main
Main.include("julscr2.jl")

So it works, but for every Julia file that I want to import to Python I have an additional Python file as an interface between both.

It there probably a more elegant way?

1 Like
9

If you have a dozen of files, it’d be useful to group them into a package. This applies to pure Python, pure Julia, and hybrid projects. You may not need to, but I’m mentioning this as you asked for a more “elegant” approach.

Note that this implies scr1 is scr2. This is how Python modules work and not PyJulia specific.

Note that include is not import (in Julia or Python sense). If you do this, once both pyjulint1 and pyjulint2 are imported, it is equivalent to

Main.eval("""
function testf_1(x)
  return x*0.1
end

function testf_1(x)
  return x*0.2
end
""")

i.e., no namespaces as you had in pyscr1.py and pyscr2.py.

1 Like
11

@tkf thank you! I am not sure I completely understand it, but it is the case. I can call

scr2.testf_1(2)

and get the same result.

12

My point is that the namespace is not separated. I think you get it.

13

I don’t think scr1 is scr2 they just don’t have anything in their namespaces. So instead of identity a would say it is equality. (like two empty sets are equal)

I mean python modules are working in this way only in this specific situation (when modules are effectively empty).

Or am I missing something? (I suppose (mayby wrong) that importing julia twice create only one julia subprocess…)

14

Here “identity” and also “equality” are technical terms about data model (see, e.g., 3. Data model — Python 3.8.1 documentation). You need to understand the difference between is and == in Python (which is like === and == in Julia). If you google “python is vs ==” you’ll find a lot of articles explaining the concept.

I’m not sure what you mean by “this specific situation” but scr1 is scr2 is true even if Main is not empty. In fact, src1, src2 and julia.Main are all same object (namely the Python module julia.Main).

15

You understood me well and you are right! Sorry for my mistake and thanks for your explanation!

As also python doc describe, fully qualified names of packages are used in import mechanism and are imported only once. (means scr1 and scr2 are really bound to same object if they import same fully qualified name of package)

2 Likes
16

The workflow I used:

import julia
jl = julia.Julia()
jl.eval('using MyPackage') # exports foo
foo = jl.eval('foo')
jl.eval('include("my_file.jl")') # defines bar
bar = jl.eval('bar')
baz = jl.eval("""
    function baz(x,y,z)
        return x*y+z
""")
i = 42
a = np.array([1, 2, 3, 4])
foo(i, a)

Calling Julia functions from Python works for primitive parameters (including lists and dictionaries) + return types and for numpy arrays. The latter it is very efficient: even numpy arrays with GB size gets passed between Python and Julia very quickly (indeed, just a pointer is passed, the data is not copied).

2 Likes
17

I heard one of my friend said that I should make the programe more modulized.

Make the functions as REST API functions, that all languages can can get the functions, no matter what language is behind that API.

Is that possible? Is that easy to achieve?

19

For python check out the flask package, and for julia check out Genie. It’s super simple to spin up rest servers. This would belong to the larger concept of micro-services architecture, so programs written in different languages can talk to each other easily.
If you have two pieces of code that need to talk to each other, you want to make sure if one program is down (for whatever reason), it can restart and/or the other program has better error handling for such cases. Typically this can be easily accomplished by containerize the code in a docker that runs non-stop ( there are flags for this ), or do a while true bash script that runs the code.