Define Python function in Julia file... with package imports

[gdalle]

Thanks to PythonCall.jl, Python functions can be defined in Julia files like so:

julia> using PythonCall

julia> @pyexec """
       def fib(n):
           a, b = 0, 1
           for i in range(n):
               a, b = b, a+b
           return a
       """ => fib
Python: <function fib at 0x7fc57dbf9940>

julia> fib(10)
Python: 55

But for some reason it fails with package imports. Note that numpy is indeed present in the environment, as evidenced by the print statement:

julia> @pyexec """
       import numpy as np
       print(np)
       def mysum(x):
           return np.sum(x)
       """ => mysum
<module 'numpy' from '[...]/.CondaPkg/env/lib/python3.11/site-packages/numpy/__init__.py'>
Python: <function mysum at 0x7fc57dbf96c0>

julia> mysum([1, 2])
ERROR: Python: NameError: name 'np' is not defined
Python stacktrace:
 [1] mysum
   @ /home/guillaume/Work/GitHub/Julia/Sandbox/pythoncall_jax.jl:38:4
Stacktrace:
 [1] pythrow()
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/err.jl:94
 [2] errcheck
   @ ~/.julia/packages/PythonCall/wXfah/src/err.jl:10 [inlined]
 [3] pycallargs
   @ ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:210 [inlined]
 [4] pycall(f::Py, args::Vector{Int64}; kwargs::@Kwargs{})
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:228
 [5] pycall
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/abstract/object.jl:218 [inlined]
 [6] (::Py)(args::Vector{Int64})
   @ PythonCall ~/.julia/packages/PythonCall/wXfah/src/Py.jl:341
 [7] top-level scope
   @ ~/Work/GitHub/Julia/Sandbox/pythoncall_jax.jl:45

@cjdoris any clue about this one?

See earlier post where @hubert had asked the same question at the end:

Define Python function in Julia file with PythonCall?

Context: I have a Pluto notebook for teaching where I need to create a Python function. I want to use PythonCall for better portability, but I miss the py"..." syntax of PyCall which allows you to directly define Python functions inside Julia code. Is there a workaround to importing the function from a file? I looked at @py but it doesn’t seem to work for my case.

[cjdoris]

Ok. So. @pyexec and @pyeval work in a local scope, but this scope only exists while the code is being executed, it is not captured by any functions you define - this just seems to be how exec works in python.

It’s not just imports that it affects, it’s all variables outside of the function scope. This includes the function itself so you can’t define recursive functions like this either.

One solution is to define the things you need to capture as globals. In your case you can put global np at the top. This is probably what most people want to do.

Another solution (if you really need to avoid polluting global scope) is to put it all inside another function, which creates a new scope that can be captured, like:

@pyexec """
def makemysum():
    import numpy as np
    ...
    return mysum
mysum = makemysum()
"""

It’s not glamorous but it works.

Note: When I say “global scope”, I’m referring to a Python scope which is unique to whichever Julia module you are running in, so it’s not truly global. You don’t need to worry about code from outside your module interfering with Python’s global scope.

[gdalle]

thanks!