# Wrap Julia package in Python: which is the best option?

**URL:** <https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344>\
**Category:** Package Management\
**Tags:** python\
**Created:** [April 11, 2022, 1:46pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344 "2022-04-11T13:46:54Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [April 11, 2022, 1:46pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/1 "2022-04-11T13:46:54Z")

</div>

I have written some modules in Julia that I’d like to call from Python.  
While this is not an issue per se, I’d like to be able to call those packages _without_ having Julia installed. Although this may look as a weird request, I want to share my codes with other people I am working with…that mainly works with Python.

After a quick discussion with @miguelraz , I have considered several options:

1. Using [JuliaCall](https://cjdoris.github.io/PythonCall.jl/dev/juliacall/) . From the description, it looks that it could handle automatically the installation of Julia if no local version is found.
2. Make a Python wrapper, such as in [diffeqpy](https://github.com/SciML/diffeqpy) (@miguelraz suggested me to tag @ChrisRackauckas )
3. Make a Python wrapper, such as in @MilesCranmer 's [PySr](https://github.com/MilesCranmer/PySR). If I correctly understand, the conda forge recipy automatically installs Julia if required.

I know that installing Julia is not difficult, but I’d like to make at least a module that is pip installable and does not require manual Julia installation, to start spreading Julia with older peeps in my group.

I thank you in advance for your suggestions.  
Cheers,  
Marco

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [April 11, 2022, 3:33pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/2 "2022-04-11T15:33:51Z")

</div>

I would like @tkf to help with getting [https://github.com/SciML/diffeqpy/pull/100](https://github.com/SciML/diffeqpy/pull/100) finished, as that would probably be the “best” way in some sense of “best”. Right now we’re in a few decent but only okay local optima.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [April 12, 2022, 3:46am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/3 "2022-04-12T03:46:04Z")

</div>

I also don’t know what the best solution is, but I’ll share my experience:

I’ve had a really good overall experience using conda-forge to manage the installation of PySR–which is itself a scikit-learn Python frontend to [SymbolicRegression.jl](https://github.com/MilesCranmer/SymbolicRegression.jl).

Users of the package–many of whom have never even heard of Julia–don’t need to worry about ever touching Julia themselves; everything (Julia binary + PyJulia) is installed automatically by conda. Since conda installs a full-blown version of Julia, to update/install packages I call `Pkg` normally from inside PySR and have an environment installed which is specific to the version of the package. For example, PySR v0.7.9 would have its julia deps under a shared environment `@pysr-0.7.9` within conda’s Julia registry–PySR itself creates this environment upon running `pysr.install()`.

The [Julia “feedstock”](https://github.com/conda-forge/julia-feedstock) is well-maintained (thanks to the hard work of @mkitti and @ngam!), and to create my package I only need to list Julia and PyJulia as dependencies in this [build file](https://github.com/conda-forge/pysr-feedstock/blob/main/recipe/meta.yaml). You would submit a file like this to the conda-forge repo and they generate your feedstock repo automatically.

Now, the major downside of this is that you need to use conda which is very heavyweight (although it is nice to have a nearly _[hermetic](https://docs.bazel.build/versions/main/hermeticity.html)_ environment despite using two separate languages!). In practice I actually have both conda and pip setup, and just let the user decide whether they want to install Julia manually for pip, or just conda.

---

<div class="post-metadata">

**Author:** ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)\
**Post date:** [July 28, 2022, 9:44pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/4 "2022-07-28T21:44:18Z")

</div>

I missed this post till now. Later this summer we will be returning to `julia_project`. Finishing the diffeqpy application is a priority. I opened a PR for diffeqpy, but `julia_project` has evolved quite a bit since then, so it needs to be refreshed.

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 5, 2022, 10:56am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/5 "2022-08-05T10:56:01Z")

</div>

@MilesCranmer , quick question - how does pysr get around pyjulia’s [issue with statically linked Python executables](https://pyjulia.readthedocs.io/en/stable/troubleshooting.html#your-python-interpreter-is-statically-linked-to-libpython).

---

<div class="post-metadata">

**Author:** ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)\
**Post date:** [August 5, 2022, 4:35pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/6 "2022-08-05T16:35:09Z")

</div>

> [@marcobonici](#):
>
> I know that installing Julia is not difficult, but I’d like to make at least a module that is pip installable and does not require manual Julia installation

I have been working on some tools to help with this. I’ll have time later this summer to get back to it (as I said above). I really think we need modular/composable tools because people will package their projects in different ways that we haven’t thought of.

[find\_julia](https://github.com/jlapeyre/find_julia). This is a Python library that can be used to search for a Julia installation on your machine. Optionally, it will automatically download and install one and then return the path to the executable. It can detect juliaup installations, but uses jill.py to install. As juliaup becomes more capable, I can add more support for it.

[julia\_project\_basic](https://github.com/jlapeyre/julia_project_basic) Given the path to a Julia project (say `Project.toml` file) it checks if `Manifest.toml` is written and is up to date. It will instantiate the project if needed. It also checks for and installs registries if you specify them. It will try to see if your PyCall.jl is in good shape and fix it.

[julia\_project](https://github.com/jlapeyre/julia_project) This builds on the previous package. You specify a few details of your Julia project inside your Python package. This tries to automate every aspect of installing Julia and packages. This includes compiling and using system images. The idea is to have all of this without requiring that the end user think about Julia.

[CachePath](https://github.com/jlapeyre/CachePath.jl) This can be used with PyCall to avoid having to rebuild PyCall every time you switch libpythons. It requires a small patch to PyCall. I have a PR up that needs work to complete.

[julia\_semver](https://github.com/jlapeyre/julia_semver) This implements completely the julia semver semantics and syntax that Pkg uses. It translates to that used by a Python library. This is used in the other packages.

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 9, 2022, 10:53pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/7 "2022-08-09T22:53:34Z")

</div>

> [@oschulz](#):
>
> how does pysr get around pyjulia’s [issue with statically linked Python executables](https://pyjulia.readthedocs.io/en/stable/troubleshooting.html#your-python-interpreter-is-statically-linked-to-libpython).

@ChrisRackauckas does SciML/diffeqpy have a way around the “statically linked Python” problem?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 10, 2022, 12:46am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/8 "2022-08-10T00:46:53Z")

</div>

Not yet

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 10, 2022, 7:53am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/9 "2022-08-10T07:53:27Z")

</div>

Darn. 🙂 (Thanks for the info!)

---

<div class="post-metadata">

**Author:** ![dmolina](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmolina/32/5246_2.png) [@dmolina](https://discourse.julialang.org/u/dmolina)\
**Post date:** [August 10, 2022, 11:39am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/10 "2022-08-10T11:39:43Z")

</div>

Thank you for these packages. I think it is a great idea to automatically install julia binary and libraries.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [August 20, 2022, 1:37am UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/11 "2022-08-20T01:37:11Z")

</div>

Sorry for the late reply. I get around it here: [PySR/julia\_helpers.py at e29a6dab162d0cf476fa274aaa0c8c16f418c8f8 · MilesCranmer/PySR · GitHub](https://github.com/MilesCranmer/PySR/blob/e29a6dab162d0cf476fa274aaa0c8c16f418c8f8/pysr/julia_helpers.py#L75-L104)

Basically just call

```python
Main = init_julia()

```

rather than

```python
from julia import Main

```

and it will implement the workaround for static python binaries.

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 20, 2022, 1:23pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/12 "2022-08-20T13:23:15Z")

</div>

> [@MilesCranmer](#):
>
> I get around it here: [PySR/julia\_helpers.py at e29a6dab162d0cf476fa274aaa0c8c16f418c8f8 · MilesCranmer/PySR · GitHub](https://github.com/MilesCranmer/PySR/blob/e29a6dab162d0cf476fa274aaa0c8c16f418c8f8/pysr/julia_helpers.py#L75-L104)

Thanks! Is there any drawback to that (and if not, is this a path to fixing that annoying issue in general)?

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 20, 2022, 1:25pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/13 "2022-08-20T13:25:34Z")

</div>

> [@oschulz](#):
>
> Thanks! Is there any drawback to that (and if not, is this a path to fixing that annoying issue in general)?

Ah wait - you do disable compiled modules in that case though, right?

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 20, 2022, 3:51pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/14 "2022-08-20T15:51:30Z")

</div>

There’s one more new option. Not sure It’s been announced, but it installs from Python (and its Julia sub-module TyPython.jl was recently registered too):

> **[GitHub - Suzhou-Tongyuan/jnumpy: Writing Python C extensions in Julia within 5...](https://github.com/Suzhou-Tongyuan/jnumpy)**
>
> Writing Python C extensions in Julia within 5 minutes.

That should work for all Julia code (also juliacall, part of PythonCall.jl). There’s also, but still only for special Julia code:

> [@Successful Static Compilation of Julia Code for use in Production](https://discourse.julialang.org/t/successful-static-compilation-of-julia-code-for-use-in-production/79318):
>
> The [StaticCompiler](https://github.com/tshort/StaticCompiler.jl) package was recently registered. This post records a successful experiment to statically compile a piece of Julia code into a small .so library on Linux, which is then loaded from Python and used in training of a deep learning model. TLDR Static compilation to a stand-alone library does work on Linux but has rather significant restrictions on the functionality available. It is mostly useful for core computational routines. Roughly speaking, if it could be implemented in plai…

one of its limitations:

> Doesn’t currently work on Windows.

and building on it (with same (above) limitations):

> [@\[ANN\] StaticTools.jl: Enabling StaticCompiler.jl-based compilation of Julia code to standalone native binaries by avoiding GC allocations and \`llvmcall\`-ing all the things](https://discourse.julialang.org/t/ann-statictools-jl-enabling-staticcompiler-jl-based-compilation-of-julia-code-to-standalone-native-binaries-by-avoiding-gc-allocations-and-llvmcall-ing-all-the-things/80398):
>
> Technically this one has been registered for a couple months already, but we never made an announcement, so here goes! [GitHub - brenhinkeller/StaticTools.jl: Enabling StaticCompiler.jl-based compilation of (some) Julia code to standalone native binaries by avoiding GC allocations and llvmcall-ing all the things!](https://github.com/brenhinkeller/StaticTools.jl) I’ve just released version 0.3, which brings a lot of major changes. While it’s still a somewhat experimental package, I think it’s at least to some extent ready for other folks to use,…

> Calling compiled Julia library from Python

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [August 20, 2022, 8:55pm UTC](https://discourse.julialang.org/t/wrap-julia-package-in-python-which-is-the-best-option/79344/15 "2022-08-20T20:55:34Z")

</div>

Yes, pre-compilation will be disabled. Depending on your package it will make startup time slower, but it won’t affect the runtime otherwise.

I used to display a warning to the user about this, but it barely affects the runtime of PySR so I just turned it off.

Honestly I’m not sure why PyJulia treats it as an error. Automatically turning off pre-compilation with a warning would be better.