Does using ccall(...) in a module prevent its precompilation?

General Usage
1

I’m getting constant warnings for one module among many:

┌ Warning: Module my_module_1 with build ID 54547777901434 is missing from the cache.
│ This may mean my_module_1 [top-level] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:941

and the only difference I can think of between that module and the others is that it is the only one which uses ccall(…) to call an external system library. Does using ccall(…) in a module prevent its precompilation? Or is it possibly because the ccall(…) passes a Ref(…) on non-constant program data?

2

No, ccall should be compiled down to a native call. Can you try disabling precompilation (__precompile__(false)) and making sure your code works and passes any tests you might have?

3

No, I’ve certainly used ccall within precompiled modules without issue. Note, however, that using ccall to, for example, initialize a pointer may not be safe during precompilation, since that pointer will not be valid later. But even that wouldn’t cause the issue you’re seeing. Can you provide a minimal example that demonstrates the problem?

4

When I try to reduce to a minimal example, I get the error message exactly once but then Julia manages to maintain the precompiled module in its cache so I don’t get it on the second or subsequent runs (unlike the behavior I see without simplification). Would this still be considered a bug?

5

How do you know that Julia manages to precompile the module the second time? If you’re talking about in-memory caching, then of course Julia can do this; any code that can be JIT compiled can be stored in memory. Of course, serializing to disk (what precompilation actually does) is the part that isn’t guaranteed to be possible for all code.

Anyway, can you provide the code you’re working with that’s causing this error? Additionally, details about your OS and Julia (via versioninfo()), and your active project (in Julia, ] status) would allow us to troubleshoot this properly without lots of hand-waving.

6

As I stated, there is no error message, and in addition the precompilation file is visible inside ~/.julia/compiled/v1.1/ .

The reason I ask is that I do not like to do a lot of work to report things which turn out to not be considered bugs. The code is crufty, and I think for the project developers’ sake it is I who should do the work to make a nice clear example.

7

It might be a bug, it might not be a bug, but we’ll never know until we see the code that causes it :smile:

1 Like
8

The problem is caused by circular module imports. Example (the modules are in separate files):

module A

using B

export f1, f2

function f1(x)
return x + A.g1(x)
end

function f2(x)
return x^2
end

end # module A

module B

using A

export g1, g2

function g1(x)
return 5 * x
end

function g2(x)
return A.f2(x) - x^3
end

end # module B

if a script uses both modules, its precompilation can never succeed.

using A
using B

print(A.f1(3) + B.g2(5), “\n”)

I suppose it is a bug that the precompiler doesn’t complain in understandable language about such situations where it isn’t smart enough to work.