[ANN] jlrs 0.18: export code written in Rust à la CxxWrap

Package Announcements
, , ,
1

jlrs is a crate for the Rust programming language that provides access to most of the Julia C API, it can be used to embed Julia in Rust applications and to use functionality it provides when writing ccallable functions in Rust.

Version 0.18 brings a new major feature to jlrs: the ability to export types and functions written in Rust to Julia. Like CxxWrap.jl, these exported items can be made available in Julia with very little code. Libraries that use this feature can be distributed as a JLL package.

Other changes mostly serve to facilitate this new feature.

Improved version and platform support

Previously jlrs only supported the stable and LTS versions of Julia, this restriction has been relaxed and jlrs now supports Julia 1.6 up to and including 1.9. A default version is no longer assumed, you must always enable a version feature to indicate what version of Julia you’re targeting.

The most important improvement regarding platform support is that macOS has finally been added to the list of supported platforms. Both embedding and binding are actively tested, but only with CI. Some other platforms, including FreeBSD, might work, but this hasn’t been tested. The only platform that fails to build with BinaryBuilder.jl, which is used to build the previously mentioned JLL packages, is 32-bit Windows.

Disentangled wrappers

A lot of things were called wrappers, even though some of those things didn’t even really wrap something. The wrappers concept has been ditched completely; it’s about Julia data. This data has a type, a layout, and is managed by Julia’s GC. Modules have been renamed and reorganized to better reflect that.

Async tasks and worker threads

If Julia 1.9 is used, jlrs can start the async runtime with additional worker threads. Async tasks must now set their thread affinity with the associated Affinity type. Accepted types are DispatchAny, DispatchMain, and DispatchWorker.

JlrsCore.jl package

In order to function correctly, jlrs depends on some code written in Julia. This code used to be included as a string an evaluated at runtime, but is now distributed as a separate package: JlrsCore.jl. This package is automatically installed if it’s not yet available if you embed Julia in a Rust application, but you can opt out of this behavior with the RuntimeBuilder.

The code of the JlrsReflect.jl package has been moved to this package, and is now available as the JlrsCore.Reflect module.

The JlrsCore.Wrap module provides two macros, @wrapmodule and @initjlrs, to make items exported by a library written in Rust available in Julia.

Type constructors

It’s now possible to construct a Julia type object from a Rust type using the ConstructType trait. This trait allows for arbitrarily complex type objects to be constructed as long as they can be expressed as a type that implements this trait in Rust.

Type constructors have been added for all abstract types from the Base and Core modules. The bindings generated by JlrsCore.Reflect implement this trait when no type parameters are elided, otherwise another binding is generated that doesn’t elide any type parameters and implements this trait.

Type constructors for all abstract DataTypes and UnionAlls from the Base and Core modules have been added to the types module, and can be generated with JlrsCore.Reflect for other abstract types.

Opaque types

There are now two ways that a Rust type can be exposed to Julia: either as an opaque or a foreign type. The main difference is that an opaque type can’t contain references to Julia data, while a foreign type can and requires implementing a custom mark function. To make a type exposable as an opaque type, all you need to do is implement the OpaqueType marker trait.

jlrs-macros crate

The jlrs-derive crate has been replaced with jlrs-macros. In addition to the derive macros, it provides a julia_version and julia_module macros.

The julia_version macro can be used to conditionally compile code depending on the targeted Julia version. For example, the following function is only available is Julia 1.7 or 1.8 is targeted:

    #[julia_version(since = "1.7", until = "1.8")]
    fn example() { }

The julia_module macro generates an initialization function that is called by the code generated by the @wrapmodule macro from JlrsCore.Wrap. This initialization function creates DataTypes for all exported opaque and foreign types, exposes global and constant data, and returns enough information to generate functions that call the exported Rust functions.

This macro has a many more useful features:

  • Exported items can be documented with the #[doc] attribute, this documentation is available in the Julia REPL and can be included in documentation generated with Documenter.jl.

  • Exported items can be renamed, and the new name can end in an exclamation mark to indicate that it modifies its arguments. Existing functions in other modules can be extended by renaming them to their fully qualified name.

  • All arguments of an exported function must implement CCallArg and its return type must implement CCallReturn. These traits are used to create fully-typed function signatures and ccall invocations. Multiple functions can be exported with the same name as long as their argument types are different.

  • Methods of opaque and foreign types can be exported directly, the extern "C" function necessary to call this from Julia is generated automatically. These generated functions track self to ensure no aliasing rules are broken.

  • You can write functions that dispatch their work to a thread pool, the generated Julia code uses an AsyncCondition to wait for the dispatched work to complete.

This macro can be used as follows:

    julia_module! {
        // An initialization function named `example_init_fn` is generated
        become example_init_fn;
        
        // This struct must implement `OpaqueType` or `ForeignType`
        #[doc = "    MyExportedType"]
        #[doc = ""]
        #[doc = "A type implemented in Rust"]
        struct MyExportedType;
        
        // Export a constructor for `MyExportedType`
        in MyExportedType fn new(value: i32) -> TypedValueRet<MyExportedType> as MyExportedType;
        
        // Export a method that takes a (mutable) reference to `self`, such a method must return a
        // `RustResultRet` because tracking `self` can fail. This result is automatically
        // unwrapped on success, while an exception is thrown on failure.
        in MyExportedType fn increment(&mut self) -> RustResultRet<Nothing> as increment!;
        
        // When `self` is taken by value, `self` is tracked and cloned. 
        in MyExportedType fn get_cloned(self) -> RustResultRet<i32>;
    }

The crate must be compiled as a shared system library by setting the lib type to cdylib in the crate’s Cargo.toml file. After compiling it, it can be loaded in Julia like this:

    module Example
    using JlrsCore.Wrap
    
    @wrapmodule("./path/to/libexample", :example_init_fn)
    
    function __init__()
        @initjlrs
    end
    end

For more information how to use this macro you can find the documentation here. I’ve been using this macro to write bindings for RustFFT; you can find the Rust bindings here, the build recipe for BinaryBuilder.jl in the Yggdrasil repo, and the RustFFT.jl package here.

Docs.rs
Crate
Github

16 Likes