# Creating a module with python dependencies

**URL:** <https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078>\
**Category:** General Usage\
**Tags:** python, package-compiler\
**Created:** [May 18, 2023, 5:52pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078 "2023-05-18T17:52:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ignacio\_Loaiza](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ignacio_loaiza/32/50039_2.png) [@Ignacio\_Loaiza](https://discourse.julialang.org/u/Ignacio_Loaiza)\
**Post date:** [May 18, 2023, 5:52pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/1 "2023-05-18T17:52:53Z")

</div>

Hi everyone!

I wrote some Julia code that I’m trying to convert into a module so I can interface it with some python code through JuliaCall. My module uses PythonCall to run some python code. I am currently using the lines:

```julia
UTILS_DIR = pwd()*"/src/UTILS/"
sys.path.append(UTILS_DIR)
ham = pyimport("ham_utils")
...

```

to include my python module “ham\_utils.py”. This works fine when I run the code, but when I try to precompile my module with `Pkg.precompile()` I get the error

```julia
ModuleNotFoundError("No module named 'ham_utils'")

```

I am wondering how I can fix this, since I don’t know how to tell the precompilation that the UTILS folder should be added to the path. Thanks so much for your help!

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [May 18, 2023, 6:20pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/2 "2023-05-18T18:20:58Z")

</div>

Be happy that you got an error instead of a crash. `pyimport`s must not be done by precompilation because that would store a stale memory pointer. See [Guide · PythonCall & JuliaCall](https://cjdoris.github.io/PythonCall.jl/stable/pythoncall/#Precompilation) for the right solution.

---

<div class="post-metadata">

**Author:** ![Ignacio\_Loaiza](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ignacio_loaiza/32/50039_2.png) [@Ignacio\_Loaiza](https://discourse.julialang.org/u/Ignacio_Loaiza)\
**Post date:** [May 18, 2023, 6:42pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/3 "2023-05-18T18:42:24Z")

</div>

I’ve been looking at the PythonCall guide but I don’t really understand how to fix my error. I have tried instead of importing the full module just importing individual functions as stated in the guide:

```julia
UTILS_DIR = pwd()*"/src/UTILS/"
sys = pyimport("sys")
sys.path.append(UTILS_DIR)
bar = pyimport("ham_utils").bar
bar()

```

But I am still getting `ModuleNotFoundError`. I don’t really understand what is the problem with the `pyimport`s or how to fix this.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [May 18, 2023, 6:50pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/4 "2023-05-18T18:50:34Z")

</div>

If you follow my link you will see that you _must_ do the `pyimport`s in the module’s ` __init__ ()` function. It’s possible that you will run into similar problems when you do that but then it at least has a chance to work.

---

<div class="post-metadata">

**Author:** ![Ignacio\_Loaiza](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ignacio_loaiza/32/50039_2.png) [@Ignacio\_Loaiza](https://discourse.julialang.org/u/Ignacio_Loaiza)\
**Post date:** [May 18, 2023, 7:05pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/5 "2023-05-18T19:05:05Z")

</div>

Ok so now I tried:

```julia
const ham_ref = Ref{Py}()
function __init__ ()
	sys_ref[] = pyimport("sys")

	sys_ref[].path.append(UTILS_DIR)
	UTILS_DIR = pwd()*"/src/UTILS/"

	ham_ref[] = pyimport("ham_utils")
end

get_system = ham_ref[].get_system

```

And when I try to precompile the package I get the error `access to undefined reference`, it seems that the ` __init__ ` function never ran?

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [May 18, 2023, 8:09pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/6 "2023-05-18T20:09:43Z")

</div>

Yes, the idea of ` __init__ ()` is that it should run when the module is loaded, not when it’s precompiled. I’m not sure what you want to do with `get_system` but fundamentally all symbols you retrieve from Python will be memory pointers, which are valid within the process but will be stale if you save them to disk and later reload them in a new Julia process, which is what happens when you precompile something. Then when you try to use that stale pointer you will typically get a crash because in the new process it points to some memory address you don’t have access to or to some random garbage. Thus all Python modules and functions must be loaded from ` __init__ ` so they are valid for the running process.

If you need to run some Python code during precompilation to compute some value, which is not tied to the memory layout of the current process, that’s fine. Then I would suggest to do that from a function, to reduce the risk that you happen to leave some stale pointers in the global scope that you might accidentally call from a new process.

By the way, the reason for the `ModuleNotFoundError` is possibly that you are using `pwd()` to navigate. It’s a better idea to use `@ __DIR__ ` to obtain the directory of the source code file as a starting point for building your path.

---

<div class="post-metadata">

**Author:** ![Ignacio\_Loaiza](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ignacio_loaiza/32/50039_2.png) [@Ignacio\_Loaiza](https://discourse.julialang.org/u/Ignacio_Loaiza)\
**Post date:** [May 25, 2023, 9:22pm UTC](https://discourse.julialang.org/t/creating-a-module-with-python-dependencies/99078/7 "2023-05-25T21:22:45Z")

</div>

Seems like the `@ __DIR__ ` fixed what needed to be fixed, thanks so much! From your answer I made a bit more sense of how Julia is interfacing to Python 🙂 .

Many thanks for your help!