Good way for creating a Julia project binding a C++ project

Hello !

I’m more a C++ developer than Julia dev but years ago I made a pure Julia project and successfully deployed it on the official Julia packages. This time, I made a pure C++ project (let’s name it MyProject) for a standalone graphical application and now I want to create a MyProject.jl project to bind it.

The build system of the C++ MyProject also compiles and install a shared library exporting some C++ API. In this C++ project I also have Julia bindings on this shared library. From the Julia REPL, I can load the lib, call my Julia binding and run my graphical application. Everything works perfectly.

Here is a simple overview of the tree of my C++ project:

MyProject/src/*.[ch]pp  <-- Code source of my application
MyProject/src/julia/bindings.[ch]pp <-- Bind my C++ API to C for being used with Julia's ccall()
MyProject/src/julia/MyProject.jl <-- Julia functions calling ccall()

The MyProject.jl Julia file is not a module (does not have module MyProject export ... end). Therefore inside the REPL I have to do:

include("src/julia/MyProject.jl")

The code is like:

const my_lib = "/usr/lib/libmyproject.so"
function foo_bar()
    FooBar(ccall((:foobar, my_lib), Clonglong, ()))
end

I tried to read the code of project such as Gtk.jl but I did not get the point on how they manage the lib path depending on the operating system:

grep -rnHe "libgtk3"
src/Gtk.jl:14:const libgtk = libgtk3

grep -rnHe "dll"
gen/gtk_consts_gen.jl:6:    "G_MODULE_SUFFIX"           => :(Sys.iswindows() ? "dll" : "so"), #For "most" Unices and Linux this is "so".

Since my project is not a mainstream (yet ^^) project such as gtk+, therefore people does not have my shared library installed on their operating system. Therefore I’m looking how to create a pure Julia project binding my C++ project. What is the good way for:

  • Organizing folders of MyProject.jl ? For example can I use git submodules to include my C++ MyProject ? is Julia package manager deals (the ]) with git submodules ?
  • Have I to compile my C++ project from MyProject.jl or can I give directly the .so, .dll, .dylib shared libraries inside the project ?
  • How to search correctly the shared library.
  • My project depends on external system libraries (i.e. libsfml.so) shall the user has to install these libraries ?
  • When the user do a ] add MyProject how to tell to Julia clone with submodules and compile the project ?

Is something like can work ?

MyProject.jl/Project.toml
MyProject.jl/src/MyProject.jl <-- Julia module + exporting functions calling ccall()
MyProject.jl/thirdpart/MyProject/src/*.[ch]pp  <-- Code source of my git submodules C++ project
MyProject.jl/thirdpart/MyProject/src/julia/bindings.[ch]pp <-- Bind my C++ API to C for being used with Julia's ccall()
MyProject.jl/thirdpart/MyProject/src/julia/MyProject.jl <-- Julia functions calling ccall() without module keyword (maybe to be renamed)

And the code of MyProject.jl/src/MyProject.jl

module MyProject
include("thirdpart/MyProject/src/julia/MyProject.jl")
export ...
end

Thank you !

1 Like

Julia has a nice way of dealing with binary dependecies called BinaryBuilder.jl, it’s a crosscompiler suite that we use for all our deps. Yggdrasil hosts all the scripts and deal with their dependencies etc.

The scripts generate something called jll packages, which when loaded load the shared library and needed dependencies

1 Like

Thanks I’ll study this !

1 Like

I believe the “Julian” way of hosting bindings to external libraries is exactly what @gbaraldi mentioned.

To build upon on their answer:

You will want to create two Julia packages:

1.) myproject_jll.jl - This package is automatically created by BinaryBuilder.jl via Yggdrasil. The purpose of this package is to retrieve the library specific to the host’s platform. This is achieved using the (automatically generated) Artifacts.toml which the package manager uses to retrieve the correct binary.

2.) MyProject.jl - This package depends on myproject_jll.jl, and its purpose is to form wrappers around all the exported functions in the C++ API (via ccall and @cfunction). This is so that the user is not required to use ccall themselves, and they are given a nice Julia API to work with. This package can be manually created, or I believe Clang.jl can automate this process.

With regards to myproject_jll.jl, this package will only be automatically generated (via Yggdrasil) if the source code of MyProject is hosted on a public repository. If not, you will manually have to create myproject_jll.jl, by running the build_tarballs.jl script locally, and then manually edit the Artifacts.toml to ensure that each platform has an appropriate url to the platform-specific library.

If you follow this method, most of your questions in the bullet points will be automatically answered (thanks to BinaryBuilder.jl). The alternative method to this, is to create a /build directory, and place a build.jl to manually compile the C++ code. This method is not as seamless as the aforementioned method in my opinion.

Good luck!

3 Likes