Packaging mixed Julia and non-Julia code

General Usage
1

Let’s say I have a repository foo_app consisting of an application (written in another language) and also the Julia bindings for that application. In my mind, this application is not, itself, a Julia package, so it makes sense to call the repository foo_app rather than Foo.jl. I was wondering if you all had any recommendations for publishing the Julia bindings in such an instance.

Option 1 would be to put the Julia bindings in a new repo, Foo.jl and have them depend on foo_app as an external dependency (or a git submodule). That makes it easy to have a self-contained package of Julia bindings, but also makes my life a bit more complex since I have to manage two repos and keep them in sync.

Option 2 would be to try to format the foo_app repo so that it can also function as a Julia package. That would mean, at least, giving it a src folder containing Foo.jl and a REQUIRE file at the top level. This means only managing one repo, but it feels like kind of a mess, since it makes it hard to cleanly separate the Julia and non-Julia components.

Is there another option? I recall something from JuliaCon about Pkg3 allowing packages to live in subfolders, so perhaps I could have Foo.jl as a subfolder of foo_app?

2

If the app is also under your control and tightly coupled with the Julia code, I would keep it in a single repo, which may simplify a lot of things (eg CI).

If, on the other hand, it is conceivable that someone would want to develop bindings for the app in another language, I would put it in another repo.

1 Like
3

There are several Julia packages that may have similar need with yours. Currently most of Julia bindings use Option 1 and I have not seen any other package uses Option 2

Bind Binary Build-able package

Many Julia package have binary dependencies, I’ll just pick one:

MXNet.jl This one binds to MXNet’s CXX backend by a build.jl it will download the CXX backend and build it to a link library that callable by its Julia part in deps dir.

Bind Python

If you want to bind a exist python package to Julia, it would be better to learn it from JuliaPy projects.

PyCall.jl This one need a python interpreter and it will search its dependency first in the system and will download a miniconda if it does not exist. This will also be a package you will need.

Pandas.jl This one need you to install python’s Pandas first and then install the package.

Currently, it seems that we are not able to make a python binding that can let user install it with only one command.

Other Possible Options

IMHO, your second option is a little bit ugly. No matter what kind of language you are using, name a binding with simply src would make confusion…

Although Julia cannot search a module inside a folder with other names at the moment (like in Python you can simply use any name for the directory and install it to some env). You can use other language to install your Julia package or specific Julia to search for another dir. Because, in some situation (like IDE, editor extensions), it may not allow developers to separate their Julia scripts

For example, if your other languageg is python, let’s say your project names foo, and it may looks like

foo_repo_name
├── foo
├── julia
│   └── Foo
│       ├── README.md
│       └── src # Here is your Foo.jl pakcage for Julia bindings
├── ruby
├── setup.py
└── some_other_languages_dir

you can write a setup.py using setuptools in Python that copys your julia package to Julia’s package directory and set its dependencies to correct path (by default it is .julia/v0.x/)

For other languages, it would be similar. you could use CMake, or their own package manager and build tools (e.g npm for node, gem for ruby, etc.) to finish this job. But this is not under Julia’s scheme and there is no mature tools in other language to help you find correct path for Julia packages, you have to handle this yourself.

There are some projects use similar solution, I’ll just pick one of them: vscode julia language support

4

Thank you both! That’s very helpful. I’ve ended up going with option 1, with the main app in one repo and the Julia and Python bindings in separate repos. So far it’s working pretty well, but I’ll update this post if the system eventually becomes untenable.