Low-cost abstractions for C library wrappers

General Usage
1

Hi,

As I am experimenting several ways to abstract over C libraries wrapped with Clang with minimal overhead, I am starting to wonder whether Julia is a good fit for this kind of thing if one wants to preserve performance. The hardest thing is to make sure that pointers remain valid, particularly when C structs themselves involve pointers to other C structs. Having to fight compiler optimizations and garbage collection to keep pointed variables valid is a tough challenge, even if ccall helps for simple API calls.

I would love to have your opinion on the subject. So far, I have investigated two tricks that work well for preserving pointer validity, but which incur high runtime costs:

  • Using a global WeakKeyDict to keep Refs/Strings/Arrays alive as long as their “parent” lives
  • Tweaking the @ccall macro to:
    1. produce pointers on-the-fly upon an API call (from a Julian struct which may contain plain struct/Ref/Array/String fields) and explicitly preserve them using LLVM instructions for that specific call
    2. create the corresponding C structs (with pointers that are guaranteed to be valid)
    3. call the API function

For the first option, allocations seem inevitable when adding anything to that dict, and simple structs take as long as ~300 ns to initialize.
For the second option, simple structs initialize quickly (~15 ns which would be totally OK) but the more complex the struct is, the slower it gets (since a whole struct has to be generated at every API call). Plus, handling of optional parameters (such as when NULL can be accepted from a C function instead of a struct pointer) are nontrivial to implement without losing a lot of performance.

My desire for performance comes from the fact that the library I am trying to wrap (Vulkan, a library for doing GPU stuff including rendering graphics) would be used inside an event loop with a lot of API calls (potentially hundreds if not thousands of them) per loop.

What do you think on that subject? Is there something to fix/improve which would prevent from fighting the language wherever pointers are involved from an external C library? Or is it just something that Julia is not intended for, and therefore one should rely on other languages such as Rust or C/C++?

2

No

No there’s no need to tweak anything about @ccall.

No it shouldn’t. If there’s something that is presistent, it is not a problem at all to keep it that way. Whatever you compute on the fly is always cacheable in the object itself.

3

If you haven’t already, I would recommend looking at how https://github.com/JuliaGPU/CUDA.jl handles this kind of abstraction. I’ve not seen anyone complaining about pointer invalidation in it or https://github.com/JuliaGPU/VulkanCore.jl/, so some specific (counter)examples would be helpful.

4

Could you be more specific please? ccall preserves anything that was created using cconvert until the call returns, but when you cconvert a struct with pointers, AFAIK there’s no way to guarantee the validity of its pointer fields. If you consider the struct

struct VkInstanceCreateInfo
    sType::VkStructureType
    pNext::Ptr{Cvoid}
    flags::VkInstanceCreateFlags
    pApplicationInfo::Ptr{VkApplicationInfo}
    enabledLayerCount::UInt32
    ppEnabledLayerNames::Ptr{Cstring}
    enabledExtensionCount::UInt32
    ppEnabledExtensionNames::Ptr{Cstring}
end

then how to make sure that the Ptr fields are valid? The issue becomes even bigger when one of these pointer fields point to a struct that has pointers itself, such as pApplicationInfo in this example:

struct VkApplicationInfo
    sType::VkStructureType
    pNext::Ptr{Cvoid}
    pApplicationName::Cstring
    applicationVersion::UInt32
    pEngineName::Cstring
    engineVersion::UInt32
    apiVersion::UInt32
end

This struct has Cstring values that need to be valid, since they point to a String somewhere. So the String they originate from must also be preserved during the ccall.

I don’t know of a way to address this kind of case in the current implementation of @ccall. If there is, I would highly appreciate to learn about it. I went through the documentation several times and through the source code, so I think I have a fairly precise understanding of its behavior.

From what I see, the API calls involve simple types that are well handled with ccall. Particularly I haven’t seen structs that involve pointers to other structs, it is mostly pointers to integer-like types such as UInt, opaque pointers or arrays (in the form of a pointer) whose objects are owned by the library. The problem I am facing involve passing pointers to memory that is owned by Julia.

True, I would love to have some feedback from VulkanCore.jl users, since this is the API that is giving me hurdles. I hope someone would have a counter-example that involves a solution to my problem. Else, I may be the person complaining about it :sweat_smile: well, not actually about not knowing how to get around it, but about related performance issues.

5

There’s nothing really more specific about this. There’s just nothing you can change about @ccall. Everything it can possibly do are doable in your own code.

What’s invalid about them? What are they pointing to? You said cconvert a struct with pointer, so this struct is the input, there’s nothing to be done if you’ve already got a pointer.

Yes, as I said, you can always cache the result. Just store the String as a field of the struct returned from cconvert and keep it there. There’s no need to construct any new object during cconvert if you already have it.

1 Like
6

That did it! Problem solved, thanks :smiley:

2 Likes