`ccall` on Windows `could not load symbol. The specified procedure could not be found`

I am loading in a custom C library and then calling functions with @ccall. Everything works fine on Apple and Linux with:

hdl = Libc.Libdl.dlopen("path/to/clibrary", Libc.Libdl.RTLD_GLOBAL)
wind_module = @ccall ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}
...
Libc.Libdl.dlclose(hdl)

However, on Windows I get:

could not load symbol "ssc_module_create":
  The specified procedure could not be found. 

I saw this same error on Linux before adding Libc.Libdl.RTLD_GLOBAL to Libc.Libdl.dlopen.

I’ve scoured the internet for a solution but nothing has come up. Anybody now how to make this work on Windows?

Does the library export the symbol? For example, on Linux what’s the output of

nm /path/to/lib.so | grep ssc_module_create

?

Yes I checked that on Linux and confirmed that all symbols are exported. How would one check that all symbols are exported on Windows?

I don’t know, I’m not familiar with Windows tools, I’d use BinaryBuilder.jl for that :grinning_face_with_smiling_eyes: or also ObjectFile.jl

BTW, why if the symbol is exported you had to dlopen it with RTLD_GLOBAL?

I wish I knew why: just found it in another discourse topic as an aside, like “(on Linux you have to add RTLD_GLOBAL)”. I think that RTLD_GLOBAL is the default on Apple so it works without any flags.

That’s probably the key for this. On Windows not all symbols are exported by default from a library. Dependency Walker or Depends let you see which symbols are exported.
Other things that at times happens is that some dependency is not in Windows path and therefore can’t be found. The dependency tools mentioned above are your only friends.
Let me just add that how fuzzy this might seems finding dependency hells on Windows is much easier than on *nix.

Thank you @joa-quim!

Unfortunately I do not have access to a Windows machine, but am relying on Github Actions to test the Julia package on Windows.

I have found a work-around, but I wonder if it indicates that there is a simpler solution?

The original problem is that calling the symbols directly fails:

wind_module = @ccall ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}
could not load symbol "ssc_module_create":
  The specified procedure could not be found.

However, if I first load the symbol with dlsym then I can call it without an error:

ssc_module_create  = Libc.Libdl.dlsym(hdl, :ssc_module_create)
wind_module = @ccall $ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}

I tested this with setting and getting a parameter in the C module as well and it worked. So I guess the solution is to load all of the C functions in with dlsym?

Assuming hdl is the shared library,

@ccall hdl.ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}

Maybe this explains why you had to dlopen the library with RTLD_GLOBAL on Linux: you were calling the the function wrong

If you’re on Linux:

julia -e 'using BinaryBuilderBase; BinaryBuilderBase.runshell(Platform("x86_64", "windows"))'

gives you a shell where you can run nm on Windows shared libraries. Of course you need to install BinaryBuilderBase.jl first

1 Like

Sure enough:

sandbox:${WORKSPACE} # nm ssc.dll 
/opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-nm: ssc.dll: no symbols

Is there a way to install and run Julia in this shell from BinaryBuilder.jl? (I want to try your suggestion of hdl.ssc_module_create on Windows).

Uhm, try nm -D maybe?

No, you can’t run Windows executables from inside this shell, this is simply a small Alpine Linux environment where we have some tools for cross-compilation.

You can try using Wine: I once tried running Julia under Wine, it was an… interesting experience.

But if you can reproduce the error on GitHub Actions, you can log into the runners.

Also objdump:

sandbox:${WORKSPACE} # ${target}-objdump -t ssc.dll 

ssc.dll:     file format pei-x86-64

SYMBOL TABLE:
no symbols

and ObjectFile.jl:

julia> using ObjectFile

julia> readmeta("./ssc.dll") do oh
           symbol_name.(Symbols(oh))
       end
AbstractString[]

don’t see any symbols. Are you really sure you compiled this shared library correctly, exporting all symbols?

hdl is defined as:

hdl = Libc.Libdl.dlopen("ssc.dll")

I tried the syntax with hdl and ccall on Apple, but maybe I’m doing something wrong?

julia> hdl = Libc.Libdl.dlopen("libssc.dylib")
Ptr{Nothing} @0x00007fb4c0dcf330

julia> @ccall hdl.ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}
ERROR: TypeError: in ccall, expected Symbol, got a value of type Ptr{Nothing}

julia> @ccall $(hdl.ssc_module_create)("windpower"::Cstring)::Ptr{Cvoid}
ERROR: type Ptr has no field ssc_module_create

julia> @ccall ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}
Ptr{Nothing} @0x00007fb4c3fcec30

The last ccall works on Linux and Apple but not Windows.

julia> hdl = "./libssc.so"
"./libssc.so"

julia> @ccall hdl.ssc_module_create("windpower"::Cstring)::Ptr{Cvoid}
Ptr{Nothing} @0x000056513091d1f0

See the example of glib in the docstring of @ccall. You can ccall without specifying the library name only functions that are in the global table

I marked the ObjectFile check for exported symbols as the solution since it is the simplest way to check the exported symbols using Julia.

I did not know that you could call the library with just the path to library file. Are there any issues with memory leaks using this method as opposed to opening a handle to the library using Libc.Libdl.dlopen (and closing the handle with Libc.Libdl.dlclose)?

I did not compile the Windows dll but the person that did is asking “what compiler flags need to be set for it to export function names?” Do you know the answer to this or a source that I can review? I believe that the issue may be related to compiling with Visual Studio - but I heard that anecdotally.

No. This is what all JLL packages do: they dlopen the library at __init__ time (without RTLD_GLOBAL), and then you call functions into them by specifying the names of the libraries (e.g. ccall((:func_name, "lib_path"), ...), or the @ccall equivalent). No need to manually dlclose the libraries.

Note that Libdl is a standard library, you don’t need to qualify it with the Libc submodule of Base, but you need to add Libdl to your dependencies in case you go by this route.

I may be wrong, I’m not super familiar with Windows, but I don’t think this is a compiler flag. In some cases you may need to explicitly mark functions to be exported, see for example winapi - Exporting functions from a DLL with dllexport - Stack Overflow. The fact that compilers on Unix systems export by default all symbols doesn’t encourage me to dig into Windows-specific oddities :slightly_smiling_face:

1 Like

So… I have confirmed via logging into the Github Action runner as you suggested that the string path to the library works with ccall on Windows in a Julia REPL (also confirmed on two other Windows machines). But something is up with using ccall and the Windows .dll in a module. I have:

  1. defined the string path variable as global hdl = nothing at the module level.
  2. Then in the calling function I have global hdl = "path/to/ssc.dll" (based off of OS).
  3. But when I @ccall hdl.function_name(...) I get **ERROR:** UndefVarError: function_name not defined.

Is there something wrong with this method? (It works on Linux and Apple). The example in the documentation for calling external C libraries defines the string path as a const, but should I do that if I need to change the string path based on the OS?

Nevermind! I had a triple string block for taking notes on the ccall issues, which turns out was affecting my definition of the string path variable for Windows! Yargh!