Two issues related to `ccall`

General Usage
1

Hi there,

This is a follow-up of Issue #27477. Running Julia 1.1.0 on darwin64.

      module B
           using Cairo
           lib_path = Cairo._jl_libcairo
           test() = ccall((:cairo_version_string, lib_path),Cstring,())
       end

julia> B.test()
ERROR: TypeError: in ccall: first argument not a pointer or valid constant expression, expected Ptr, got Tuple{Symbol,String}
Stacktrace:
 [1] test() at ./REPL[1]:4
 [2] top-level scope at none:0

I found a workaround:

module B
    using Cairo
    lib_path = Cairo._jl_libcairo
    test() = eval(:(ccall((:cairo_version_string, $lib_path),Cstring,())))
end

julia> B.test()
Cstring(0x000000012c08bc49)

But when I try to to parametrize the argument type but ran into another issue. See the sample code below

julia> module B
           using Cairo
           lib_path = Cairo._jl_libcairo
           arg_type = ()
           call_str =:(ccall((:cairo_version_string, $lib_path),Cstring,$arg_type))
           dump(call_str)
           test() = eval(call_str)
       end
WARNING: replacing module B.
Expr
  head: Symbol call
  args: Array{Any}((4,))
    1: Symbol ccall
    2: Expr
      head: Symbol tuple
      args: Array{Any}((2,))
        1: QuoteNode
          value: Symbol cairo_version_string
        2: String "/Users/hcui7/.julia/packages/Homebrew/s09IX/deps/usr/lib/libcairo.dylib"
    3: Symbol Cstring
    4: Tuple{} ()
Main.B

julia> B.test()
ERROR: syntax: ccall argument types must be a tuple; try "(T,)"
Stacktrace:
 [1] eval at ./boot.jl:328 [inlined]
 [2] eval at ./REPL[10]:1 [inlined]
 [3] test() at ./REPL[10]:7
 [4] top-level scope at none:0

If I hard-code the arg_type, it works:

julia> module B
           using Cairo
           lib_path = Cairo._jl_libcairo
           arg_type = ()
           call_str =:(ccall((:cairo_version_string, $lib_path),Cstring,()))
           dump(call_str)
           test() = eval(call_str)
       end
WARNING: replacing module B.
Expr
  head: Symbol call
  args: Array{Any}((4,))
    1: Symbol ccall
    2: Expr
      head: Symbol tuple
      args: Array{Any}((2,))
        1: QuoteNode
          value: Symbol cairo_version_string
        2: String "/Users/hcui7/.julia/packages/Homebrew/s09IX/deps/usr/lib/libcairo.dylib"
    3: Symbol Cstring
    4: Expr
      head: Symbol tuple
      args: Array{Any}((0,))
Main.B

julia> B.test()
Cstring(0x0000000123892c49)

Julia>
2

FYI, this is completely unrelated.

The error should be pretty clear for this one. Your library name is not a const.

No, don’t do that.

You spliced an tuple, not an expression. This is clearly shown in your dump. And again, don’t use eval.

3

I’m sorry for my confusion and really appreciate your reply. I’m still new and struggling to learn Julia. Could you suggest fixes for my issues?

For my first issue, my lib_path is assigned programatically. I tried to make a copy of lib_path inside the function by declaring const, but it errors out

ERROR: LoadError: syntax: unsupported `const` declaration on local variable around /Users/hcui7/repos/FMUSim.jl/src/FMUSim.jl:30

For my second issue, I guess I’m still confused. Could you suggest a fix so that the eval would run? I would like to compare the dump results. I should be able to get rid of eval after the first issue is resolved.

Thanks again.

4

I got around the first issue by using dlopen and dlsym, referencing the discussion here: https://github.com/JuliaLang/julia/issues/2316

For the second issue, I found a seemingly related discussion at https://github.com/JuliaLang/julia/issues/1313. Still trying to digest and not able to get it right.

Any suggestion is much appreciated.

5

After a day’s trial and error, I still haven’t figured out how to “splice an expression” as suggested in your post. May I ask for further hints? @yuyichao

Quite a few examples I found just enumerated the symbols and wrote one Julia function for each symbol. I also found the Clang.jl package which generates wrapper functions to ccall based on the prototypes defined in headers. Although verbose, it should perform much better than the dynamic path I was heading. Please correct me if I’m wrong.

Regards,
Hantao

6

There’s nothing you need to figure out about “splice an expression”. You know how to splice things, ($ syntax) so you just need to figure out how to construct.

instead of

shown in your debugging output.

You already know that the expression you want to construct: (), and you know that to get an object that corresponds to the expression you simply quote that expression, so you just need to quote the () to get the expression you want.

julia> dump(:(()))
Expr
  head: Symbol tuple
  args: Array{Any}((0,))

You can’t. const must be global. It’s unclear why you want to create a local variable for it.

It’s very unclear what you want to do and what you want to by dynamic. For a start, there’s almost nothing dynamic about C libraries and there’s almost nothing dynamic about calling them. The only thing that’s marginally dynamic and what ppl usually mean for dynamic in this context is to load the library and look up the symbol at runtime. This is only useful if you simply don’t know about the library or function name before hand, I highly doubt that’s your issue. If it is, then please post your real case rather than the libcairo. dlopen/dlsym is only needed for this case so unless you are doing something else that you haven’t mentioned, you should not use them.

Other than this, there’s absolutely nothing dynamic about calling C functions so what do you want to achieve? Are you trying to do all your ccalls in a function? If yes, then why? Are you just looking for a different syntax for ccall? Again, why do you want to do that? What’s the issue you have with normal ccall syntax? The only case I can think of that might make you want to change the syntax a little bit is if you think the ccall syntax is too verbose. However, since the ccall syntax contains the absolute minimum information you need when making a C function call, this can only be an issue if the information you want is available elsewhere, say if you are making the function call many times in different places, or if your library follows some convension where some types can be inferred from other types or the function names, In the former case, you may want to have one julia function that calls this particular C function (similar to Clang.jl) but having a more “dynamic solution” does not help you at all (does not make it less verbose) since you still need to supply all the type information for the function. (The spelling will be different for sure but the information or the number of symbols will basically be the same). In the later case, you can indeed safe some typing by doing some computation at compile time. You can do this either with generated functions or macros depending on the kind of computation you need. Such macros or functions will look nothing like what you have (both will use metaprogramming for sure but that’s where the similarity ends…) so if you have questions about this then you need to be more specific about what you want to know. Again, a more “dynamic” version, whatever that means, will not help you at all, either in performance or in saving typing.

7

Thanks for sharing your thoughts! I’m trying to write a library in Julia for simulating Functional Mockup Units (FMUs) generated from OpenModelica. Each FMU contains a shared library and the metadata for the model. There is a list of known functions each FMU library can provide, so believe you are right, I don’t have use Libdl.

Initially, I wanted to write one wrapper function in Julia to ccall the functions provided by the library, for example

function fmi2Call(lib, func_name, ret_type, arg_type, arg_val) 
    ccall((func_name, lib), ret_type, ($(arg_type...),), arg_val...)
end

so that I can call the library functions in the following way without writing 20-wrapper functions.

fmi2Call("CoupledClutches.dylib", "fmi2GetVersion", Cstring, (), ())

Then, I found Clang.jl which can basically wrap the functions automatically with minimum tweaks required. I’m giving up on this approach as it is simple enough to write a wrapper for each function.

I also read from https://github.com/JuliaLang/julia/issues/1313#issuecomment-8940281 that
ccall evaluates its first 3 arguments at compile time, and also requires that the argument type tuple be syntactically present inside the ccall. This is why eval is often used around definitions using ccall.

The ccall in my fmi2Call function apparently does not have all first three arguments syntactically present, so even after I fixed the tuple issue, I still get errors like

julia> function fmi2Call(lib, func_name, ret_type, arg_type, arg_val)
           ccall((func_name, lib), ret_type, :($(arg_type...),), arg_val...)
       end
ERROR: syntax: ccall argument types must be a tuple; try "(T,)"

julia> function fmi2Call(lib, func_name, ret_type, arg_type, arg_val)
           ccall((func_name, lib), ret_type, :($(())), arg_val...)
       end
ERROR: syntax: more arguments than types for ccall

julia> function fmi2Call(lib, func_name, ret_type, arg_type, arg_val)
           ccall((func_name, lib), ret_type, :($(())))
       end
ERROR: could not evaluate ccall return type (it might depend on a local variable)

Anyway, I’m gonna wrap each function individually. The issue can be closed. Thanks for your time again!!

8

I mean, what’s the point for this? How is this different from,

ccall(("CoupledClutches.dylib", "fmi2GetVersion"), Cstring, ())

Yours is actually longer so I don’t see why you would want to change the syntax this way.

And what you want is impossible.

And the point is, why do you even need to wrap anything. Again, if the point is to save typing the types, then what you wanted just doesn’t do it.

Edit: just to be clear, writing or generating julia wrappers for C functions is fine. It’s just that there’s nothing dynamic needed or possible in this process.

9

Now my version looks like

function fmi2GetVersion() :: String
    ret = ccall(("CoupledClutches.dylib", "fmi2GetVersion"), Cstring, ())
    unsafe_string(ret)
end

And I can call it using fmi2GetVersion() instead of ccall(("CoupledClutches.dylib", "fmi2GetVersion"), Cstring, ()) just to save some typing.

For other functions with more arguments, the Julia wrapper function check the input values before calling ccall. Is this the case where a wrapper is needed? I need to read more and follow up later.

10

I got it. I’m convinced that I need nothing dynamic in this process.