I get errors when using module

General Usage
1

Dear Julia friends,

First of all, I apologize for my poor English skills. I have a problem developing a module that uses Embeddings.jl:

module Portal
include("./t.jl")
export get_embedding
end # module Portal

using Embeddings
const embtable = load_embeddings(FastText_Text{:de})
const get_word_index = Dict(word => i for (i, word) in enumerate(embtable.vocab))

function get_embedding(word)
    ind = get_word_index[word]
    emb = embtable.embeddings[:,ind]
    return emb
end

When using Portal I get the following error messages:
julia> using Pkg

julia> Pkg.activate(“.”)
Activating project at ~/JuliaProjects/Portal

julia> using Revise

julia> using Portal
Precompiling Portal…
lld: error: /home/fahim/.julia/compiled/v1.11/Portal/jl_ysPtHq(text#0.o):(.rodata+0x114): relocation R_X86_64_PC32 out of range: 2562091264 is not in [-2147483648, 2147483647]

referenced by text
GLOBAL_OFFSET_TABLE
referenced by dict.jl:199

lld: error: /home/fahim/.julia/compiled/v1.11/Portal/jl_ysPtHq(text#0.o):(.rodata+0x14c): relocation R_X86_64_PC32 out of range: 2562091408 is not in [-2147483648, 2147483647]

referenced by text
…
…

Stacktrace:
[1] pipeline_error
@ ./process.jl:598 [inlined]
[2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
@ Base ./process.jl:513
[3] run
@ ./process.jl:510 [inlined]
[4] link_image (repeats 2 times)
@ ./linking.jl:166 [inlined]
[5] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{…}, loadable_exts::Nothing)
@ Base ./loading.jl:3085
[6] (::Base.var"#1110#1111"{Base.PkgId})()
@ Base ./loading.jl:2579
[7] mkpidlock(f::Base.var"#1110#1111"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
@ FileWatching.Pidfile /opt/julia-1.11.3/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:95
[8] #mkpidlock#6
@ /opt/julia-1.11.3/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:90 [inlined]
[9] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
@ FileWatching.Pidfile /opt/julia-1.11.3/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:116
[10] #invokelatest#2
@ ./essentials.jl:1057 [inlined]
[11] invokelatest
@ ./essentials.jl:1052 [inlined]
[12] maybe_cachefile_lock(f::Base.var"#1110#1111"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
@ Base ./loading.jl:3698
[13] maybe_cachefile_lock
@ ./loading.jl:3695 [inlined]
[14] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:2565
[15] __require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:2388
[16] #invoke_in_world#3
@ ./essentials.jl:1089 [inlined]
[17] invoke_in_world
@ ./essentials.jl:1086 [inlined]
[18] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:2375
[19] macro expansion
@ ./loading.jl:2314 [inlined]
[20] macro expansion
@ ./lock.jl:273 [inlined]
[21] __require(into::Module, mod::Symbol)
@ Base ./loading.jl:2271
[22] #invoke_in_world#3
@ ./essentials.jl:1089 [inlined]
[23] invoke_in_world
@ ./essentials.jl:1086 [inlined]
[24] require(into::Module, mod::Symbol)
@ Base ./loading.jl:2260
Some type information was truncated. Use show(err) to see complete types.

But if I include the file and then call the method, it works.

julia> include("./src/t.jl")
get_embedding (generic function with 1 method)

julia> get_embedding("Buch")
300-element Vector{Float32}:
 -0.0781
 -0.0145
  â‹®

Thank you in advance for any tips or solutions.
Fahim

2

Hi Fahim,

Welcome to the Julia community!

On Windows 10 I get a (presumably) somewhat similar issue:

julia> using Portal
ERROR: Error opening package file (...)\.julia\compiled\v1.11\Portal\bI5Dh_Imu6F.dll: %1 is not a valid Win32 application.

Looking at the file in question, it is about 2.5 GB in size. It then seems likely that the precompiled files can be at most 2^{31} - 1 bytes (typemax(Int32) == 2147483647) for 32-bit compatibility. In this case the two consts (in particular embtable) will be too large to cache.

One option would be to disable precompilation by adding __precompile__(false) to Portal.jl. Or (and a bit better) you could use Preferences.jl to achieve the same. The downside is obviously that when using Portal you’ll need to compute embtable and get_word_index, which at least on my pc takes quite a while (for some reason). I suppose you could just manually save and load embtable.embeddings and get_word_index to and from disk.

3

Thank you for the welcome and support.

FYI:
it now works by calling the function precompile(false). But the following exception is thrown:
exception = Error when precompiling module, potentially caused by a precompile(false) declaration in the module

Best regards
Fahim

4

Yeah, it seems that __precompile__(false) disables/interrupts precompilation by making it throw an exception. With the Preferences.jl approach you can get rid of this (minor) annoyance.

1 Like