Jluna: a new Julia <-> C++ wrapper

Available here: https://github.com/Clemapfel/jluna

(Edit: the license of the library has been changed to MIT, making it freely available without restrictions)

I have spend the last few months, full-time, writing a Julia ⭤ wrapper completely from scratch. It aims to fully replace the C-API in usage in C++ projects, and offers a possible alternative to Cxx.jl.

I aim for jluna to be…

  • modern: uses the most modern C++20 features, the newest compiler versions and Julia 1.7+
  • generic: all std:: C++ and C classes are wrapped, and can be move between Julia and C++ freely. Any Julia object can be moved to C++ or access and modified directly
  • safe: full exception forwarding from julia, compile-time assertions, safety from the garbage collector, etc.
  • fast: as of version 0.7, jluna went through extensive optimization to reduce overhead compared to the C-API as much as possible
  • elegant: generic syntax and modern design patterns make C++ syntax feel almost as nice as Julias
  • well-documented: verbose documentation, a handwritten manual that introduces every feature step-by-step, installation guide, inline-documentation for IDEs
// execute arbitrary strings with exception forwarding
State::safe_script(R"(
    f(x) = x*x*x
    
    mutable struct MyStruct
        _field
        MyStruct() = new(123)
    end
    instance = MyStruct();
)");

// access and modify variables
Main["instance"]["_field"] = 456;
State::script(R"(println("instance._field is now: ", instance._field))");

// call julia-side functions with C++-side values
int result = Main["f"](12);
Base["println"](result);

/// muti-dimensional, 0-overhead array interface
State::script("jl_array = reshape(collect(1:9), 3, 3)");
Array<size_t, 2> cpp_array = Main["jl_array"];

// iterable and assignable
for (auto e : cpp_array)
    e = e.operator size_t() + 10; // this also modified julia-side variable Main.jl_array

// julia-style, multi-dimensional indexing
cpp_array.at(1, 2);

// julia-style array comperehension
cpp_array.at({8, 2, 5, 6});

Even more exciting examples can be found on github!

Please consider checking it out, it is available for free under MIT license. I am proud of my work and I hope that some others may also find it useful or at least enjoyable.


Hidden bonus: jluna made C++ have list comprehension:

using namespace jluna;
for (size_t i : "(i for i in 1:10 if i % 2 == 0)"_gen)
	std::cout << i << " ";
2 4 6 8 10 
62 Likes

how copyleft is this license? can people use it in a project without open-sourcing the rest of the project?

It seems the OSL is a strongly viral copyleft license:

Clause 1.c says

1.c) to distribute or communicate copies of the Original Work and Derivative Works to the public, with the proviso that copies of Original Work or Derivative Works that You distribute or communicate shall be licensed under this Open Software License;

Clause 5 says

  1. External Deployment. The term “External Deployment” means the use, distribution, or communication of the Original Work or Derivative Works in any way such that the Original Work or Derivative Works may be used by anyone other than You, whether those works are distributed or communicated to those persons or made available as an application intended for use over a network. As an express condition for the grants of license hereunder, You must treat any External Deployment by You of the Original Work or a Derivative Work as a distribution under section 1(c).

So this seems similar to the AGPL, in that it prevents any use in combination with proprietary code, regardless of whether that software is running as a service or is distributed to end users.

The usual uncertainties apply in the definition of “derived work”, but it’d probably be best to assume any program which uses this library also needs to be released under the OSL as soon as it is distributed or used over a network.

Just to clarify, afaik all copyleft licenses permit use of the software by end users without restriction. It’s only including it as part of service/application/library provided to a third party that requires licensing.

For example, if you are a researcher you can use the GPL-licensed GNU Coreutils without sharing your code.

1 Like

It’s worse than that. While the intent of the OSL may be similar to APGL (and GPL), the OSL is GPL-incompatible, so technically incompatible with Julia (for distribution/conveying default Julia), and any other GPL or APGL software.

Right, you can download any GPL-software, e.g. Julia (and some other, e.g. OSL or AGPL-software), and use. But you can’t distribute with OSL software (or proprietary), e.g. with help of PackageCompiler.jl except by jumping through some hoops.

You may think Julia is MIT licensed (only), but it does have at least one GPL component for sparse-matrix work (SuiteSparse), and using that part of Julia with OSL, seems prevented for sure. That’s very tragic (and not the intent), as both the GPL and the OSL are made for free software, and the FSF is ok with OSL for free software, just judges it GPL-incompatible. I’m not a lawyer but it might be all right (or not) to distribute Julia with the GPL component[s], and OSL, if it’s actually no being used (falling under “mere aggregation”).

Depending on the motives of @Clemapfel I would suggest relicensing to the updated LLVM license (in cases it’s about patents, or MPL 2.0 if also about (weak) copyleft, or even just to MIT): https://releases.llvm.org/13.0.0/LICENSE.TXT

Note, Apache License 2.0 is also one-way incompatible with GPL. LLVM adopted it as it’s new license, but with the “LLVM Exceptions” to make it GPL compatible.

2 Likes

Is it an alternative? [And CxxWrap.jl isn’t “rapidly aging” in case anyone misunderstood, and Cxx.jl is being resurrected for Julia 1.7 by @Gnimuc.]

Since, those packages are well, Julia packages, mainly for calling from Julia to C++, I believe (the latter technically to C API of C++ project that you have to make with CxxWrap). I’ve never looked seriously into calling in the other direction (well from C), as I though it easy (also not something I need) even without a package.

It’s great to have this new package to call from C++, but it seems to me mainly for calling from C++ to Julia (maybe also helps for from C?). I DO see it is bidirectional, but that still means you must embed Julia in C++ right? I.e. there is not corresponding Julia package.

See also PythonCall.jl which is, well, is to call Python, but also in the other direction, by also providing a python package.

I’m guessing this package is corresponding to Rust package (also for bidirectional calling, supporting Julia 1.7, and 1.6 with lesser support, for Linux and Windows, with “MSVC support” added “4 days ago”): https://github.com/Taaitaaiger/jlrs

I suppose since you can embed Julia in Rust, you could e.g. call from Rust to Julia, but then to C++ (with CxxWrap, but not with Jluna). Or from C++ (with Jluna) to Julia, and back to C++ (with Jluna, or even CxxWrap, but you couldn’t to Rust?).

I’m mostly concerned with these packages working, but it’s also intriguing to see: benchmarks — sol 3.2.3 documentation

Since your package is emulating sol in some way, does it also have a speed advantage over the alternatives? And the other have GC issues?

If I need to call C++, I believe @barche’s CxxWarp.jl is still a good option (on all platforms, even on Windows, which is not tested for Jluna, but should work):

the user must have a C++ compiler installed which supports C++17 (e.g. GCC 7, clang 5; for macOS users that means Xcode 9.3).

I’m not sure, but I believe that doesn’t preclude calling C++20 code, e.g. with modules. While for Jluna, since it uses C++20, I’m sure it should work, and I suppose all C++17 or older will, just with a C++20 compiler.

FYI: I see you’re using: ⭤ - Left Right Triangle-Headed Arrow: U+2B64 - Unicode Character Table (which I see as a square) so I suggest you change here (and at github) to ASCII “<->”, as the sol project does.

1 Like

I thought OSL worked differently, and have decided to just drop the issue and change the license to MIT. My only intention was for the license to have a clause that forces any derivative work to disclose the source, and I was under the impression that OSL allowed any other use without restriction. I am not very knowledge about law, so I think it is best to be on the safe site in terms of usability instead of focusing on getting credit.

40 Likes

I edited the original post because it does seem kind of mean, cxxwrap.jl is very good and while C++ made a lot of progress in the past 2 years, it isn’t notable enough to use that wording.

I was under the impression that Cxx.jl maintainance ended and that CxxWrap.jl is incompatible with Julia 1.5+. I am happy that the latter is being fixed, because I don’t think jluna and CxxWrap are interchangeable. The main difference is that jluna uses C++ as the host language while CxxWrap uses Julia so I don’t think jluna would ever fully replace it.


The similarities between sol and jluna are more architectural, for example both have a central proxy class that automatically decays on assigning and can hold any value (which, in lua, is a table) and the syntax of both is very similar.
Performance-wise, it is actually kind of hard to evaluate jluna. The only good comparison I was able to make was to the C-API. I tried my very best to minimize the overhead compared to if you were to do it using only C.

If you allocate a julia value directly through C, the GC needs to be manually made aware that it is still in usage, otherwise it’ll swipe the memory from under you while you’re still using it C-side. The main reason for jluna was because I got so frustrated with that, and I wanted something that automatically protects only what needs to be protected.

2 Likes

The former is being fixed (if you meant write that), the latter is not being fixed, it seems already fixed with 1.6 in the compat section. I’ve only tried Cxx.jl minimally, at the time, which I believe still fully works under Julia 1.3, so “aging”, but has a Draft PR for supporting Julia 1.7 (not 1.6).

1 Like

This is a nice change, and very much in line with what most of the major Julia packages use. Thank you for reconsidering the license choice!

2 Likes

Thanks for that change, it will majorly reduce legal concerns and increase the usage of the package (which is very cool). Btw, you still have the old license file in the repo as LICENSE.txt along with the new license in just LICENSE. Probably best to remove the old file to avoid confusion.

8 Likes

Awesome work! The interface seems very intuitive and I’m looking forward to trying out your package :+1:

Is Julia code executed as efficiently as if it were run from pure Julia run?

Also, kudos for taking feedback so constructively and integrating it right away!

3 Likes

thank you for pointing that out, I just fixed it

I have a section in the manual about this (chapter 15, performance), basically there are two ways to call julia code:
a) like you would in the REPL, by evaluating a string which then causes a julia-side action. This version does have overhead because Meta.parse and eval have to be called and you need to move the string.
b) by directly triggering memory pointing to a julia function to be called, like you would call a c function. This version has 0 overhead, it is exactly as fast as if the entire call was in pure julia.

Performance in jluna is always a trade-off between convenience and speed. I usually focus on convenience and safety with the main classes. I still try my best to make these fast as possible, but there are also 0-overhead versions of almost all features, especially those that are often performance critical like handling big arrays or basic things like calling functions. These are less pretty, less convenient, but make sure to let julia be the racecar that it is.

2 Likes

Yes, CxxWrap.jl works on Julia 1.6 and 1.7.

Jluna looks quite promising, it’s always good to have options. If one size fit all in computing, we wouldn’t be here :wink:

9 Likes

From a personal point of view, this is great!

I think viral licenses like OSL can be useful in certain contexts. For example, if you’re trying to build a business on open software and don’t want a large SAAS competitor to come in and take all the customers after you’ve done all the hard software work. Or for open source teams who are building a “product-like” software project and similarly don’t want a commercial entity to “embrace and extend” their product, fracturing the community of users and unfairly profiting from the product.

On the other hand, MIT/BSD style licenses are great for very generic libraries — they maximize “freedom to reuse / compose” in any context. This is pretty important when software projects are built from hundreds of libraries with long dependency chains. If a library with a viral license is part of such a project it can “infect” the whole project, even though it’s only a very small part of the whole!

Having said all that, I respect people’s choice of license!

The MIT license does include the clause

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

So it does require attribution if someone copies and redistributes the code.

The big problem is in defining “derivative work” — if this is interpreted broadly then any software using the library counts as a derivative work, regardless of whether the library itself has been modified and the modified source made available. One could argue against this on a case-by-case basis, but people are usually not willing to take this risk.

4 Likes

Thanks a lot for sharing, this is really interesting work!

I have one question, in the README you list as future work:

thread-safety

Do you think this will require changes in the core of Julia itself? The last time I checked it was not possible to instantiate two different Julia contexts in the same process.

1 Like

I have a question about the boxing of strings. When a C++ std::string object is passed to the Julia side, does it involve a deep copy of the array of characters in the string? In other words, can I call a string-processing Julia function from C++ without copying the underlying data?

No, the data is not copied, only a pointer to the std::string object is passed. The Julia AbstractString interface is implemented by calling C++ functions on that pointer, meaning each call is expensive compared to calling a native Julia function (accessing a character is a C++ function call!). Performance may be better if you do explicitly copy the data, if memory allows.