Identical dependency being replaced while importing packages

General Usage
1

I am facing an issue that seems to have cropped up suddenly. I am trying to import the packages JLD2 and NamedArrays on a HPC cluster (not sure if this is relevant), and they seem to clash over DataStructures. I’m not sure what’s going on and how to resolve this.

If I import JLD2 before NamedArrays I get

julia> import JLD2

julia> import NamedArrays
┌ Warning: Replacing module `DataStructures`
└ @ Base loading.jl:947

whereas if I import them by reversing the sequence I obtain

julia> import NamedArrays

julia> import JLD2
[ Info: Precompiling JLD2 [033835bb-8acc-5ee8-8aae-3f567f8a3819]
┌ Warning: Module DataStructures with build ID 7498601554111257 is missing from the cache.
│ This may mean DataStructures [864edb3b-99cc-5e75-8d2d-829cb0a9cfe8] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1016
[ Info: Skipping precompilation since __precompile__(false). Importing JLD2 [033835bb-8acc-5ee8-8aae-3f567f8a3819].

This is not happening on my local computer. I am running this on a login node on a single processor, so there’s no race condition.

The relevant lines in loading.jl appear to be

key = PkgId(m, String(nameof(m)))
    if haskey(loaded_modules, key)
        oldm = loaded_modules[key]
        if oldm !== m
            @warn "Replacing module `$(key.name)`"
        end
    end

so I check what’s going on

julia> using JLD2

julia> key = Base.PkgId(JLD2.DataStructures, String(nameof(JLD2.DataStructures)))
DataStructures [864edb3b-99cc-5e75-8d2d-829cb0a9cfe8]

julia> haskey(Base.loaded_modules,key)
true

julia> oldm = Base.loaded_modules[key]
DataStructures

julia> using NamedArrays
┌ Warning: Replacing module `DataStructures`
└ @ Base loading.jl:947

julia> m = Base.loaded_modules[key]
DataStructures

julia> m == oldm
false

Why are the modules different? And how can I solve this (hopefully without removing the compile caches altogether, but I’m ok with that if there’s no choice)?

1 Like
2

It appears to have been related to the compile cache. Removing ~/.julia/compiled/v1.4 and precompiling the packages has resolved the issue. Not sure what exactly was resulting in this.