Building and Calling a Shared Library

General Usage
1

Hey!

Is it possible to build and call a shared library file with Julia?

What I tried is the following:
I’ve got a file called mylib.jl with the following content:

module MyLib

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    return 0
end

Base.@ccallable function myfun(ARGS::Vector{String})::Cint
    return 1
end

end

Then I’m compiling an executable along with a shared library file by running:

julia> jcp = joinpath(Pkg.dir("PackageCompiler"), "juliac.jl")
"/Users/lackner/.julia/v0.6/PackageCompiler/juliac.jl"

julia> run(`julia $jcp -vae mylib.jl`)
WARNING: Your Julia system image is not compiled natively for this CPU architecture.
        Please run `PackageCompiler.force_native_image!()` for optimal Julia performance
Julia program file:
  "/Users/lackner/Documents/Julia/Scripts/mylib.jl"
C program file:
  "/Users/lackner/.julia/v0.6/PackageCompiler/src/../examples/program.c"
Build directory:
  "/Users/lackner/Documents/Julia/Scripts/builddir"
Change to build directory
Build module image files ".ji" in directory "/Users/lackner/Documents/Julia/Scripts/builddir":
  `/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/bin/julia -Ccore2 -J/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/lackner/Documents/Julia/Scripts/builddir")) # enable usage of precompiled files
        include("/Users/lackner/Documents/Julia/Scripts/mylib.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`
Build object file "mylib.o" in directory "/Users/lackner/Documents/Julia/Scripts/builddir":
  `/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/bin/julia -Ccore2 -J/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no --output-o /Users/lackner/Documents/Julia/Scripts/builddir/mylib.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/lackner/Documents/Julia/Scripts/builddir")) # enable usage of precompiled files
        include("/Users/lackner/Documents/Julia/Scripts/mylib.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`
Build shared library "mylib.dylib" in build directory:
  `cc -m64 -shared -o mylib.dylib /Users/lackner/Documents/Julia/Scripts/builddir/mylib.o -std=gnu99 -I/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-install_name,@rpath/mylib.dylib`
Building executable "mylib" in build directory:
  `cc -m64 '-DJULIAC_PROGRAM_LIBNAME="mylib.dylib"' -o mylib /Users/lackner/.julia/v0.6/PackageCompiler/src/../examples/program.c mylib.dylib -std=gnu99 -I/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-rpath,@executable_path`
/Users/lackner/.julia/v0.6/PackageCompiler/src/../examples/program.c:32:62: warning: incompatible pointer types passing 'jl_datatype_t *' (aka 'struct _jl_datatype_t *') to parameter of
      type 'jl_value_t *' (aka 'struct _jl_value_t *') [-Wincompatible-pointer-types]
    jl_array_t *ARGS = jl_alloc_array_1d(jl_apply_array_type(jl_string_type, 1), 0);
                                                             ^~~~~~~~~~~~~~
/Users/lackner/Apps/Julia-0.6.app/Contents/Resources/julia/include/julia/julia.h:1199:58: note: passing argument to parameter 'type' here
JL_DLLEXPORT jl_value_t *jl_apply_array_type(jl_value_t *type, size_t dim);
                                                         ^
1 warning generated.
All done

This produces the files mylib, mylib.o and mylib.dylib in the ./builddir folder.

I can run the executable without an error:

shell> cd builddir
/Users/lackner/Documents/Julia/Scripts/builddir

shell> ./mylib

Works!

But if I’m trying to call the shared library I get an error and Julia crashes:

julia> ccall((:myfun, "mylib"), Int64, ())

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
unknown function (ip: 0xffffffffffffffff)
Allocations: 2013324 (Pool: 2011978; Big: 1346); GC: 1
Segmentation fault: 11

Is it even possible to achieve what I’m trying to do? Ultimately I want to implement that shared library in a LabView program.

4 Likes
2

I agree it would be nice to be able to do this with PackageCompiler. This is the issue I created for it:
https://github.com/JuliaLang/PackageCompiler.jl/issues/53

The shared library you get now has the code for julia_main, but that is not enough, julia should also be initialised for instance. Right now this happens in the executable that is compiled from program.c (or your own C code).

1 Like
3

This is pretty old now but I just came across it and in case it’s of use to anyone I thought I’d post that the OP’s problem was that external programs calling julia need to initialize the julia runtime before calling Julia functions. A nice new bit of functionality was added to PackageCompiler in https://github.com/JuliaLang/PackageCompiler.jl/pull/122. That PR adds the option to have the boilerplate code that initializes julia added to your shared lib so you don’t have to write it yourself on the external program side. You can access the functionality through the PackageCompiler api functions by passing the keyword argument init_shared = true. Note that you can always add the boilerplate yourself, so this is a convenience feature that’s been added to PackageCompiler, not a bugfix.

2 Likes