Using OrdinaryDiffEq.jl as a native ODE solver library from Rust with JuliaC --trim

I didn’t find an exemple for this so I’m posting this here just in case someone needs it. People kept saying OrdinaryDiffEq is compatible with JuliaC but I wanted to see how it looks like and if someone ask its here now. Also, didn’t want to code an adaptative alg nor learn rust libs for ode solvers.

julia :

using OrdinaryDiffEqSSPRK
using SciMLBase
using SciMLLogging

Base.errormonitor(t::Task) = t

Base.@ccallable function solve_ode(
    pout::Ptr{Float64},
    capacity::UInt64,
    pf::Ptr{Cvoid},
    pu0::Ptr{Float64},
    u0len::UInt64,
    t0::Float64,
    T::Float64,
    pp::Ptr{Cvoid},
    nsave::UInt64
)::Int64
    if u0len == 0 || nsave < 2 ||
       u0len > typemax(Int) || nsave > typemax(Int) ||
       !isfinite(t0) || !isfinite(T) || T <= t0
        return -1
    end
    rhs(du, u, p, t) = ccall(
        pf, Cvoid,
        (Ptr{Float64}, Ptr{Float64}, UInt64, Ptr{Cvoid}, Float64),
        pointer(du), pointer(u), u0len, p, t
    )
    u0 = unsafe_wrap(Vector{Float64}, pu0, (Int(u0len),))
    ode = ODEProblem{true, SciMLBase.FullSpecialize}(
        rhs, u0, (t0, T), pp
    )
    sol = solve(
        ode, SSPRK432(),
        saveat=range(t0, T; length=Int(nsave)),
        verbose=SciMLLogging.None(),
        abstol=1e-12,
        reltol=1e-12
    )
    N = length(sol.u)
    if UInt64(N) > capacity ÷ u0len
        return -2
    end
    for i in 1:N
        for j in 1:Int(u0len)
            unsafe_store!(
                pout,
                sol.u[i][j],
                (i - 1) * Int(u0len) + j
            )
        end
    end
    return Int64(N)
end

wrapper rust :

use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, Ordering};
use std::panic::{catch_unwind, AssertUnwindSafe};
#[link(name = "ssplib", kind = "dylib")]
unsafe extern "C" {
    fn solve_ode(
        pout: *mut f64,
        capacity: u64,
        pf: *const c_void,
        pu0: *mut f64,
        u0len: u64,
        t0: f64,
        t1: f64,
        pp: *const c_void,
        nsave: u64,
    ) -> i64;
}
struct Context<'a, P, F> {
    params: &'a P,
    rhs: F,
    panicked: AtomicBool,
}
extern "C" fn trampoline<P, F>(
    du: *mut f64,
    u: *const f64,
    len: u64,
    pp: *mut c_void,
    t: f64,
)
where
    P: Sync,
    F: Fn(&mut [f64], &[f64], &P, f64) + Sync,
{
    let ctx = unsafe { &*(pp as *const Context<'_, P, F>) };
    let n = len as usize;
    if ctx.panicked.load(Ordering::Relaxed) {
        unsafe { std::ptr::write_bytes(du, 0, n) };
        return;
    }
    let result = catch_unwind(AssertUnwindSafe(|| {
        let input = unsafe {
            std::slice::from_raw_parts(u, n)
        }.to_vec();
        let output = unsafe {
            std::slice::from_raw_parts_mut(du, n)
        };
        (ctx.rhs)(output, &input, ctx.params, t);
    }));
    if result.is_err() {
        ctx.panicked.store(true, Ordering::Relaxed);
        unsafe { std::ptr::write_bytes(du, 0, n) };
    }
}

pub fn solve<P, F>(
    u0: &[f64],
    params: &P,
    tspan: (f64, f64),
    nsave: usize,
    rhs: F,
) -> Result<Vec<Vec<f64>>, &'static str>
where
    P: Sync,
    F: Fn(&mut [f64], &[f64], &P, f64) + Sync,
{
    let capacity = u0.len()
        .checked_mul(nsave)
        .ok_or("Dimension error")?;
    let mut initial = u0.to_vec();
    let mut output = vec![0.0_f64; capacity];
    let ctx = Context {
        params,
        rhs,
        panicked: AtomicBool::new(false),
    };
    let n = unsafe {
        solve_ode(
            output.as_mut_ptr(),
            capacity as u64,
            trampoline::<P, F> as *const () as *const c_void,
            initial.as_mut_ptr(),
            initial.len() as u64,
            tspan.0,
            tspan.1,
            &ctx as *const _ as *const c_void,
            nsave as u64,
        )
    };
    Ok(output
        .chunks_exact(u0.len())
        .map(|row| row.to_vec())
        .collect())
}

main rust

mod solver;
use solver::solve;
struct Params {
    alpha: f64,
    beta: f64,
    gamma: i32,
}
fn rhs(du: &mut [f64], u: &[f64], p: &Params, _t: f64) {
    for i in 0..u.len() {
        du[i] = -p.alpha * u[i] + p.beta;
    }
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
    let u0 = vec![1.0, 2.0, 3.0];
    let params = Params {
        alpha: 2.0,
        beta: 0.5,
        gamma: 3,
    };
    let sol = solve(&u0, &params, (0.0, 1.0), 101, rhs)?;
    println!("Solution finale : {:?}", sol.last().unwrap());
    let exact: Vec<f64> = u0.iter()
        .map(|x| {
            params.beta / params.alpha
                + (x - params.beta / params.alpha)
                * (-params.alpha).exp()
        })
        .collect();

    println!("Solution exacte : {:?}", exact);
    Ok(())
}

compile :

juliac --output-lib ssplib --project="." ssprk.jl --trim
ln -sf ssplib.dylib libssplib.dylib
RUSTFLAGS="-L native=$(pwd) -C link-arg=-Wl,-rpath,$(pwd)" cargo run --release

and you get an adaptive ode solver from julia in rust with very similar api than julia and any kind of parameter (c representable).

One thing a little sad : Base.errormonitor(t::Task) = t hope this gets avoided at some point. Btw I didn’t put the tols in but its straighforwar and left as an exercise for the readers lol.
edit : sorry for the symbolic link just change the name rust wants libsomething.dylib.

Interesting work! How big is the lib binary?

exciting! I find it ironic that we now have “low level static languages” calling “high level dynamic” ones (pass me the tongue in cheek definitions)

Without hand removal of blas and unlinked dylib it’s 30mb after it’s 8.9mb, I’m talking about the full release folder ready to be send to another Mac and run without Julia install.

Yes but if you look at the Julia code a lot of things have a low level flavour in order to represent C

Thanks for the numbers. Do you mean the BLAS is a separate file in the folder?

Most high level languages have libraries locked in their ecosystems. Python and Matlab have a lot of useful packages, but it is hard to use them from other languages. In contrast, libraries written in C or Fortran can be used by almost all other langs. This post shows a nontrivial example that Julia has the ability to generate libraries of the same kind. I think this is very meaningful.

yes, basically at the end you have

yolhan@Mac testrs % ls lib  
julia                   libjulia.1.14.0.dylib   libjulia.1.14.dylib     libjulia.dylib          ssplib.dylib
yolhan@Mac testrs % ls share
julia
yolhan@Mac testrs % ls share/julia
cert.pem
yolhan@Mac testrs % ls lib/julia  
libatomic.1.dylib               libgmp.dylib                    libopenblas64_.dylib            libstdc++.6.dylib
libatomic.dylib                 libgomp.1.dylib                 libopenlibm.4.0.dylib           libstdc++.dylib
libblastrampoline.5.16.0.dylib  libgomp.dylib                   libopenlibm.4.dylib             libunwind.1.0.dylib
libblastrampoline.5.dylib       libjulia-internal.1.14.0.dylib  libopenlibm.dylib               libunwind.1.dylib
libblastrampoline.dylib         libjulia-internal.1.14.dylib    libpcre2-8.0.dylib              libunwind.dylib
libdSFMT.dylib                  libjulia-internal.dylib         libpcre2-8.dylib                libz.1.3.2.dylib
libgcc_s.1.1.dylib              libmpfr.6.dylib                 libquadmath.0.dylib             libz.1.dylib
libgfortran.5.dylib             libmpfr.dylib                   libquadmath.dylib               libz.dylib
libgfortran.dylib               libopenblas64_.0.3.34.dylib     libssp.0.dylib                  libzstd.1.dylib
libgmp.10.dylib                 libopenblas64_.0.dylib          libssp.dylib                    libzstd.dylib

its a lot less than full julia (without --trim) but some can still be remove like all the libblas basically which for now JuliaC doesn’t know how to proove its not used.

yolhan@Mac testrs % du -sh ./lib/julia/*
108K    ./lib/julia/libatomic.1.dylib
  0B    ./lib/julia/libatomic.dylib
  0B    ./lib/julia/libblastrampoline.5.16.0.dylib
2,5M    ./lib/julia/libblastrampoline.5.dylib
  0B    ./lib/julia/libblastrampoline.dylib
 68K    ./lib/julia/libdSFMT.dylib
176K    ./lib/julia/libgcc_s.1.1.dylib
2,1M    ./lib/julia/libgfortran.5.dylib
  0B    ./lib/julia/libgfortran.dylib
492K    ./lib/julia/libgmp.10.dylib
  0B    ./lib/julia/libgmp.dylib
444K    ./lib/julia/libgomp.1.dylib
  0B    ./lib/julia/libgomp.dylib
3,3M    ./lib/julia/libjulia-internal.1.14.0.dylib
  0B    ./lib/julia/libjulia-internal.1.14.dylib
  0B    ./lib/julia/libjulia-internal.dylib
528K    ./lib/julia/libmpfr.6.dylib
  0B    ./lib/julia/libmpfr.dylib
 11M    ./lib/julia/libopenblas64_.0.3.34.dylib
  0B    ./lib/julia/libopenblas64_.0.dylib
  0B    ./lib/julia/libopenblas64_.dylib
176K    ./lib/julia/libopenlibm.4.0.dylib
  0B    ./lib/julia/libopenlibm.4.dylib
  0B    ./lib/julia/libopenlibm.dylib
844K    ./lib/julia/libpcre2-8.0.dylib
  0B    ./lib/julia/libpcre2-8.dylib
356K    ./lib/julia/libquadmath.0.dylib
  0B    ./lib/julia/libquadmath.dylib
 72K    ./lib/julia/libssp.0.dylib
  0B    ./lib/julia/libssp.dylib
3,7M    ./lib/julia/libstdc++.6.dylib
  0B    ./lib/julia/libstdc++.dylib
 92K    ./lib/julia/libunwind.1.0.dylib
  0B    ./lib/julia/libunwind.1.dylib
  0B    ./lib/julia/libunwind.dylib
172K    ./lib/julia/libz.1.3.2.dylib
  0B    ./lib/julia/libz.1.dylib
  0B    ./lib/julia/libz.dylib
768K    ./lib/julia/libzstd.1.dylib
  0B    ./lib/julia/libzstd.dylib

and thats like half the size of the final lib.
And yes its pretty cool we can make “small” standalone dylib from julia and call it anywhere.

I did fortran here too with and without trimming and with Deep learning packages Embedding Julia code in Fortran - #14 by yolhan_mannes

I am surprised. Does trimming work for auto differentiation libraries?

Not sure, maybe Enzyme.jl but would need to test, note that you won’t be able to diff through a ccall anyway so its only for function fully predefined in julia. For instance allowing rust side activation would break the autodiff.

trimming doesn’t work with Enzyme but no trimming seems to work

using Enzyme

function juliafunc(x)
    return sum(x)
end

Base.@ccallable function cjuliafunc(px::Ptr{Float64},xlen::UInt64)::Float64
    x = unsafe_wrap(Vector{Float64},px,(xlen,))
    juliafunc(x)
end

Base.@ccallable function cjuliagrad(pg::Ptr{Float64},px::Ptr{Float64},xlen::UInt64) ::Cvoid
    g = unsafe_wrap(Vector{Float64},pg,(xlen,))
    x = unsafe_wrap(Vector{Float64},px,(xlen,))
    fill!(g,0.0)
    Enzyme.autodiff(Enzyme.Reverse,juliafunc,Duplicated(x,g))
    return Cvoid()
end
#[link(name = "ssplib", kind = "dylib")]
unsafe extern "C" {
    fn cjuliafunc(
        px: *mut f64,
        xlen: usize
    ) -> f64;
    fn cjuliagrad(
        pg: *mut f64,
        px: *mut f64,
        xlen: usize
    ) -> ();
}
fn main() {
    let mut x = vec![0.1;100];
    let s = unsafe {
        cjuliafunc(x.as_mut_ptr(), x.len())
    };
    println!("{}",s);
    let mut g = vec![0.0;100];
    unsafe {
        cjuliagrad(g.as_mut_ptr(),x.as_mut_ptr(), x.len())
    };
    println!("{:?}",g);
}
juliac --output-lib ssplib --project="." ssprk.jl  
ln -sf ssplib.dylib libssplib.dylib     
RUSTFLAGS="-L native=$(pwd) -C link-arg=-Wl,-rpath,$(pwd)" cargo run --release

gives

10
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

however since no trimming if I bundle its pretty big

juliac --output-lib ssplib --project="." ssprk.jl   --bundle
yolhan@Mac testrs % du -sh ./lib
526M    ./lib
yolhan@Mac testrs % du -sh ./share
 29M    ./share

you can with ForwardDiff but its forward mode

using ForwardDiff

function juliafunc(x)
    return sum(x)
end

Base.@ccallable function cjuliafunc(px::Ptr{Float64},xlen::UInt64)::Float64
    x = unsafe_wrap(Vector{Float64},px,(xlen,))
    juliafunc(x)
end

Base.@ccallable function cjuliagrad(pg::Ptr{Float64},px::Ptr{Float64},xlen::UInt64) ::Cvoid
    g = unsafe_wrap(Vector{Float64},pg,(xlen,))
    x = unsafe_wrap(Vector{Float64},px,(xlen,))
    cfg = ForwardDiff.GradientConfig(
        juliafunc, x, ForwardDiff.Chunk{1}()
    )
    ForwardDiff.gradient!(g, juliafunc, x, cfg)
    return Cvoid()
end

juliac --output-lib ssplib --project="." ssprk.jl --bundle --trim

yolhan@Mac testrs % du -sh ./lib
 41M    ./lib
yolhan@Mac testrs % du -sh ./share
576K    ./share

Thanks. Given that Enzyme is a complicated system, I am not surprised it cannot be trimmed. It is good to know the ForwardDiff works with trimming. That is already very useful.

Just for fun, I’m curious @yolhan_mannes what happens if you do Enzyme.autodiff(Enzyme.set_abi(Enzyme.Reverse, Enzyme.InlineABI), juliafunc, Duplicated(x, g))

chatgpt on the thousands lines build log :

The compilation fails with 4,449 errors and 5 warnings, primarily caused by incompatibilities between Enzyme’s compilation machinery and JuliaC’s trimming process.

Main issues

  1. Enzyme’s differentiation machinery cannot be fully resolved. The first Enzyme-related error (#43) occurs at Enzyme.Compiler.thunk(...), which returns Any and cannot be resolved by JuliaC’s trimming verification.
  2. Dynamic compiler infrastructure. Enzyme relies on LLVM-based code generation to produce derivatives. JuliaC’s --trim requires all reachable function calls to be statically resolvable. Enzyme’s internal compilation machinery introduces calls that the trimmer cannot resolve, producing thousands of errors involving Enzyme.Compiler, LLVM.jl, and GPUCompiler.
  3. Additional initialization errors. Some failures originate from GPUCompiler.__init__() and REPLExt.__init__(), suggesting that unrelated package initialization code is also being retained during trimming.

InlineABI doesn’t solve the problem of calling Enzyme.Compiler.thunk(…) and since it doesn’t remove error or add any it seems Enzyme ABI search is actually quite trimmable its all the rest thats not.

no issues without trimming.
btw : since it works without trimming you think I could make Enzyme diff a rust function from its llvm ? :') may not be the same version of llvm though and no point since Enzyme works in rust nightly but it would be fun, julia only see a CVoid pointer but well we never know.