Calling C Code with Automatic Differentiation in Julia

Specific Domains Numerics
, , ,
1

Hi all,

I’m reaching out to see if anyone in the Julia community has experience or advice on calling C code from Julia in a way that still supports automatic differentiation (AD).

My motivation

I’m working on a project that relies on an extensively developed C library. Rewriting this codebase in Julia is unfortunately not feasible, but I would like to call into it from Julia and still benefit from automatic differentiation to speed up downstream computations.

What I’ve tried

I’ve experimented a bit with Julia’s native @ccall interface (docs), which works well for straightforward interop. However, as expected, it doesn’t seem to play nicely with AD out of the box.

I tried this simple example:

double square(double x)
{
    return x * x;
}

with

using ForwardDiff

const lib = "./mylib.so"

square(x) = ccall((:square, lib), Cdouble, (Cdouble,), x)

println(square(3.0))                            # 9.0
println(ForwardDiff.derivative(square, 3.0))    # LoadError

Since I’m relatively new to automatic differentiation, I’d love to hear if:

  • There are Julia packages or tools that can enable AD through external C calls
  • There are recommended patterns or workarounds for integrating AD with foreign function calls

Any pointers, examples or guidance would be very much appreciated! Thanks in advance!

2 Likes
2

If you can compile the external library with clang and include llvm bytecode in the object file, Enzyme.jl is able to differentiate through that. Hopefully @vchuravy @wsmoses can provide concrete examples for that.

2 Likes
3

My naive attempt

julia> using Enzyme

julia> file = """
       double square(double x) {
           return x * x;
       }
       """;

julia> run(pipeline(`clang -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`; stdin=IOBuffer(file)))
Process(`clang -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`, ProcessExited(0))

julia> csquare(x::Cdouble) = @ccall "./libsquare.dylib".square(x::Cdouble)::Cdouble
csquare (generic function with 1 method)

julia> first(first(autodiff(Reverse, csquare, Active, Active(3.14))))
ERROR:
No reverse pass found for ejlstr$square$./libsquare.dylib
 at context:   %3 = call double @"ejlstr$square$./libsquare.dylib"(double %0) #5, !dbg !14

Stacktrace:
 [1] csquare
   @ ./REPL[16]:1

failed, so yeah, I hope they can provide a working example :slightly_smiling_face:

Edit: I found this thread Is it possible to use autodiff on an external program? - #2 by wsmoses which suggests the same strategy, but without a working example.

Edit 2: I managed not to get an error with

julia> using Enzyme, Libdl

julia> file = """
       double square(double x) {
           return x * x;
       }
       """;

julia> run(pipeline(`clang -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`; stdin=IOBuffer(file)))
Process(`clang -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`, ProcessExited(0))

julia> fptr = dlsym(dlopen("./libsquare.dylib"), :square)
Ptr{Nothing} @0x0000000104833f9c

julia> csquare(x::Cdouble) = ccall(fptr, Cdouble, (Cdouble,), x)
csquare (generic function with 1 method)

julia> first(first(autodiff(Reverse, csquare, Active, Active(3.14))))
0.0

but the result is still wrong, so I’m clearly still missing something.

4

If Enzyme is not working, I recommend differentiating your C project using the source-to-source tool Tapenade, and then ccall the differentiated routines from Julia.

5

In general, you have to write a custom differentiation rule for those calls. (with Enzyme rules or ChainRules.jl or others, depending on what AD package you are using).

1 Like
6

It appears julia changed its FFI runtime calls since last someone tried, but in any case on latest Enzyme.jl (0.13.48) and Julia 1.10 or 1.11, this should work:

(base) wmoses@hydra:~/git/Enzyme.jl$ julia +1.10 --project 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.9 (2025-03-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Enzyme
       # Enzyme.Compiler.DebugLTO[] = true

julia> file = """
              double square(double x) {
                  return x * x;
              }
              """;

julia> run(pipeline(`/home/wmoses/llvms/llvm15/buildD/bin/clang -Xclang -no-opaque-pointers -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`; stdin=IOBuffer(file)))
Process(`/home/wmoses/llvms/llvm15/buildD/bin/clang -Xclang -no-opaque-pointers -x c - -fPIC -fembed-bitcode -shared -o libsquare.dylib`, ProcessExited(0))

julia> csquare(x::Cdouble) = @ccall "./libsquare.dylib".square(x::Cdouble)::Cdouble
csquare (generic function with 1 method)

julia> @show first(first(autodiff(Reverse, csquare, Active, Active(3.14))))
first(first(autodiff(Reverse, csquare, Active, Active(3.14)))) = 6.28
6.28

Note that you need the version of clang to be less than or equal to the version of LLVM used by Julia (e.g. 15 for Julia 1.10 or 16 for 1.11).

7

Thanks everyone for the suggestions. I am glad to see that there are options I can explore.

I have been trying the suggested Julia commands, but I am having problems with the -fembed-bitcode flag:

ld: warning: -bitcode_bundle is no longer supported and will be ignored

which leads to an error in the AD:

ERROR: LoadError: 
No reverse pass found for square
 at context:   %4 = call double @square(double %0) #5, !dbg !14

I suspect this may be a known limitation of Macs. Is there a workaround for this?

I am using LLVM version 15.0.7 on a MacBook Pro M3 (installed with Homebrew).