I have written some modules in Julia that I’d like to call from Python.
While this is not an issue per se, I’d like to be able to call those packages without having Julia installed. Although this may look as a weird request, I want to share my codes with other people I am working with…that mainly works with Python.
After a quick discussion with @miguelraz , I have considered several options:
Using JuliaCall . From the description, it looks that it could handle automatically the installation of Julia if no local version is found.
Make a Python wrapper, such as in @MilesCranmer 's PySr. If I correctly understand, the conda forge recipy automatically installs Julia if required.
I know that installing Julia is not difficult, but I’d like to make at least a module that is pip installable and does not require manual Julia installation, to start spreading Julia with older peeps in my group.
I thank you in advance for your suggestions.
Cheers,
Marco
I would like @tkf to help with getting https://github.com/SciML/diffeqpy/pull/100 finished, as that would probably be the “best” way in some sense of “best”. Right now we’re in a few decent but only okay local optima.
I also don’t know what the best solution is, but I’ll share my experience:
I’ve had a really good overall experience using conda-forge to manage the installation of PySR–which is itself a scikit-learn Python frontend to SymbolicRegression.jl.
Users of the package–many of whom have never even heard of Julia–don’t need to worry about ever touching Julia themselves; everything (Julia binary + PyJulia) is installed automatically by conda. Since conda installs a full-blown version of Julia, to update/install packages I call Pkg normally from inside PySR and have an environment installed which is specific to the version of the package. For example, PySR v0.7.9 would have its julia deps under a shared environment @pysr-0.7.9 within conda’s Julia registry–PySR itself creates this environment upon running pysr.install().
The Julia “feedstock” is well-maintained (thanks to the hard work of @mkitti and @ngam!), and to create my package I only need to list Julia and PyJulia as dependencies in this build file. You would submit a file like this to the conda-forge repo and they generate your feedstock repo automatically.
Now, the major downside of this is that you need to use conda which is very heavyweight (although it is nice to have a nearly hermetic environment despite using two separate languages!). In practice I actually have both conda and pip setup, and just let the user decide whether they want to install Julia manually for pip, or just conda.
I missed this post till now. Later this summer we will be returning to julia_project. Finishing the diffeqpy application is a priority. I opened a PR for diffeqpy, but julia_project has evolved quite a bit since then, so it needs to be refreshed.
I have been working on some tools to help with this. I’ll have time later this summer to get back to it (as I said above). I really think we need modular/composable tools because people will package their projects in different ways that we haven’t thought of.
find_julia. This is a Python library that can be used to search for a Julia installation on your machine. Optionally, it will automatically download and install one and then return the path to the executable. It can detect juliaup installations, but uses jill.py to install. As juliaup becomes more capable, I can add more support for it.
julia_project_basic Given the path to a Julia project (say Project.toml file) it checks if Manifest.toml is written and is up to date. It will instantiate the project if needed. It also checks for and installs registries if you specify them. It will try to see if your PyCall.jl is in good shape and fix it.
julia_project This builds on the previous package. You specify a few details of your Julia project inside your Python package. This tries to automate every aspect of installing Julia and packages. This includes compiling and using system images. The idea is to have all of this without requiring that the end user think about Julia.
CachePath This can be used with PyCall to avoid having to rebuild PyCall every time you switch libpythons. It requires a small patch to PyCall. I have a PR up that needs work to complete.
julia_semver This implements completely the julia semver semantics and syntax that Pkg uses. It translates to that used by a Python library. This is used in the other packages.
There’s one more new option. Not sure It’s been announced, but it installs from Python (and its Julia sub-module TyPython.jl was recently registered too):
That should work for all Julia code (also juliacall, part of PythonCall.jl). There’s also, but still only for special Julia code:
one of its limitations:
Doesn’t currently work on Windows.
and building on it (with same (above) limitations):
(Thanks for the info!)