module MyModule
using PrecompileTools
export TV
@compile_workload begin
include("foo.jl") #Defines TV
end
end
When I run using MyModule, I have printouts that indicate that the contents of foo.jl run successfully as part of the include process. However, once include("foo.jl") finishes, I receive the following error.
ERROR: Failed to precompile MyModule [top-level] to "/Users/pluie/.julia/compiled/v1.9/jl_Dr5RS9".
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base ./loading.jl:2300
[3] compilecache
@ ./loading.jl:2167 [inlined]
[4] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1805
[5] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1660
[6] macro expansion
@ ./loading.jl:1648 [inlined]
[7] macro expansion
@ ./lock.jl:267 [inlined]
[8] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1611
[9] top-level scope
@ ./timing.jl:273 [inlined]
[10] top-level scope
@ ./REPL[2]:0
I have confirmed that include("foo.jl") runs without error separate from its use in this module. Does anyone have an idea of what the source of this error may be or what I can do to produce a more meaningful error description? TIA.
I’ve never seen @compile_workload surround an include call before, I would’ve expected @compile_workload to precompile the include call, not run it, so TV wouldn’t be assigned to. Does it work if you remove the @compile_workload begin block and just directly run the include?
I have @compile_workload around an include call in other modules and it works fine. However, I went ahead and tried it without the @compile_workload. I got the same error.
Any other suggestions?
Edit: I should add that what happens in include("foo.jl") requires a lot of memory. My guess is the “save to disk” portion of pre-compilation is failing, but it would be helpful to verify this and better understand what the limits are.
Good to know, and it rules out @compile_workload specifically. The error seems clear that it’s a precompilation failure, but maybe you could make certain by turning it off in a new process and seeing if the import works
It is sometimes helpful during module development to turn off incremental precompilation. The command line flag --compiled-modules={yes|no} enables you to toggle module precompilation on and off. When Julia is started with --compiled-modules=no the serialized modules in the compile cache are ignored when loading modules and module dependencies. More fine-grained control is available with --pkgimages=no , which suppresses only native-code storage during precompilation. Base.compilecache can still be called manually. The state of this command line flag is passed to Pkg.build to disable automatic precompilation triggering when installing, updating, and explicitly building packages.
I’ve never heard of a memory limit, and since disk is usually much larger than RAM I would assume (with no basis) that if your RAM can handle it then it should be cacheable. But there are some things that should be initialized with each runtime instead of precompiled and cached, but it’s not clear which, if any, would throw the error you have. It might be crucial for people to see what foo.jl does.
I started Julia using julia --compiled-modules=no and then executed using MyModule. It completed running without error.
The purpose of foo.jl is to include several functions (formed from Symbolics.build_function in a separate process), execute them for the first time, then save their handle to an array (TV) to be used later.
include("indicesPerm.jl")
include("symLists.jl")
using Symbolics
ord = 4
## include all files
(goodSet,sumIJKM) = type1Ind(ord)
TV = []
println("start include")
rstWVLxyzDict,rstDictBuild,rCoefSyms,sCoefSyms,tCoefSyms=dictSyms(goodSet,sumIJKM,ord+1)
for ii= 1:length(rstDictBuild)
fname = string("solns/order$(ord)/",rstDictBuild[ii],".jl")
if isfile(fname)
println(fname)
fh2 = include(fname)
m = first(methods(fh2))
x00 = zeros(m.nargs-1)
fh2(x00...);
push!(TV, fh2)
end
end
I’ll note that I use a similar foo.jl for a smaller set of functions (also found from using Symbolics.build_function), which works in MyModule without an issue. That’s what has led me to believe it is a memory issue.
If it’s the exact same code but it just iterates over fewer rstDictBuild, then yes it’s reasonable to conclude it’s a memory problem. It’d be nice to find out how much memory gets used up throughout these processes (without or with @compile_workload, small or large set of functions), to verify if the error is thrown right after hitting some memory limit. Logging Sys.free_memory(), Sys.total_memory(), and Sys.total_physical_memory() before, throughout, and afterwards could help, but there might be easier ways of monitoring memory usage.
The other versions of MyModule that include foo.jl use a smaller, different set of functions defined by rstDictBuild.
I re-ran using MyModule with the logging as you suggested. At the beginning (before @compile_workload) the total memory and total physical memory were both 17179869184. The free memory was 131940352. The free memory generally decreased throughout the include process (sometimes it increased between iterations but only slightly). At the end, the free memory was 272760832. This time during the save to disk process, a segmentation fault occurred (rather than the error I have previously shown).
Am I reading this right? Your free memory starts at 131940352 (which is a really low 0.768% of your total memory of 16 GiB), decreases throughout the include("foo.jl"), but then jumps up to a higher 272760832 (which would indicate the GC ran a cycle). And getting a segfault instead of a precompile failure error when the only change is printing the amount of free memory over time, it doesn’t seem like it’s as simple as just running out of working memory.
First, what’s your exact Julia version? It would be important to know if someone wants to figure out what those line numbers in your stacktrace correspond to.
Second, is there really nothing more to that error message than what you showed?
Third, that’s an unconventional (and possibly creative) use of PrecompileTools. I’m not sure what I think about it; it does make it hard to know what tasks will be precompiled and which ones won’t. Typically I leave definitions (methods, consts, etc) out of the @compile_workload and just run code from within the @compile_workload block. The only thing I see now that might be “wrong” with this approach is that you may cache more than you really want: for example, if any of the code in foo.jl is generated by metaprogramming (macros, etc.), then you’ll also be caching all the code that’s only needed to define the methods. (Typically, you only want to cache the items that the package needs to run.) This may bloat your cache files and increase loading times somewhat.
Correct, I pasted the entirety of the error message.
foo.jl does not use any metaprogramming (if I understand that correctly). The point of foo.jl is just to include and initialize the functions that were created previously by Symbolics.build_function, whose filenames are recreated by using rstDictBuild. The .jl files included and initialized are just mathematical expressions.
I’ll give Revise.track(Base) a go. But yes, unfortunately, the files foo.jl includes and initializes are too many and too large to include on this post.
[93082] signal (11.2): Segmentation fault: 11
in expression starting at REPL[4]:1
jl_crc32c at /Users/pluie/.julia/juliaup/julia-1.9.2+0.aarch64.apple.darwin14/lib/julia/libjulia-internal.1.9.dylib (unknown line)
jl_restore_package_image_from_stream at /Users/pluie/.julia/juliaup/julia-1.9.2+0.aarch64.apple.darwin14/lib/julia/libjulia-internal.1.9.dylib (unknown line)
Allocations: 7297647 (Pool: 7295376; Big: 2271); GC: 11
zsh: segmentation fault julia
The segfault occurs during CRC computation when a module is being loaded. That’s really surprising. Can you open a Julia issue? There may not be a lot we can do about this without more detail about what’s in foo.jl, but if there’s any way you can do a make debug build-from-source, that would greatly improve the quality of stack traces and the ability to guess what’s gone wrong.
Check out the julia repo and build it locally (see https://github.com/JuliaLang/julia/blob/master/README.md#building-julia). But instead of running make, run make debug. Supply full stacktraces of any Julia errors or segfaults. You can link back to this discussion for any further info.