Dynamic include file selection with world age constraints in Julia 1.12 - alternatives to conditional include?

General Usage
,
1

Hi everyone,

I’m facing a challenge with Julia 1.12’s stricter world age semantics. I have a pattern where I conditionally include different implementation variants based on configuration, but this no longer works in Julia 1.12.

Here’s a minimal example of what I’m trying to achieve:

function include_switch(a_or_b)
    mktempdir() do dir
        write(joinpath(dir, "fileA.jl"), "fooA() = :a")
        write(joinpath(dir, "fileB.jl"), "fooB() = :b")
        func = if a_or_b == :a
            include(joinpath(dir, "fileA.jl"))
            fooA
        else
            include(joinpath(dir, "fileB.jl"))
            fooB
        end
        # foo is registered in my modular system for later use, e.g. 
        # register_substep_imp(:prepare_sim, func, PRIO_NORMAL)
    end
end        

include_switch(:a)

This gives me:

WARNING: Detected access to binding `Main.fooA` in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.

The context is a modular model system where different variants can be selected through configuration. The invokelatest suggestion doesn’t work here since I’m not calling the function yet - I just want to register it in my system for later use.

What’s the recommended approach for this kind of dynamic file selection pattern in Julia 1.12?

Thanks for any insights!

2

eval messing with global names from inside methods wasn’t good practice prior to 1.12 either, const variables are a bit more obvious:

1.11.7 
julia> function foo(i)
         eval(:(const a = $i))
         a
       end
foo (generic function with 1 method)

julia> foo(0)
0

julia> foo(1)
WARNING: redefinition of constant Main.a. This may fail, cause incorrect answers, or produce other errors.
1

julia> foo(2)
WARNING: redefinition of constant Main.a. This may fail, cause incorrect answers, or produce other errors.
2

julia> foo(3)
WARNING: redefinition of constant Main.a. This may fail, cause incorrect answers, or produce other errors.
3
1.12.1 
julia> function foo(i)
         eval(:(const a = $i))
         a
       end
foo (generic function with 1 method)

julia> foo(0)
WARNING: Detected access to binding `Main.a` in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
To make this warning an error, and hence obtain a stack trace, use `julia --depwarn=error`.
0

julia> foo(1)
0

julia> foo(2)
1

julia> foo(3)
2

Function names are const too, but you’ve been getting away with this because evaluating more method definitions for the same name is not a reassignment. The problem here is the conditional existence of global names, and it’s clashing with 1.12’s tighter implementation of them. If you want to treat modules as Dict{Symbol, Any}, you need to reference the module AND use @invokelatest to access the uncertain name:

julia> function bar(i)
         eval(:(b = $i))
         @invokelatest (@__MODULE__).b
       end
bar (generic function with 1 method)

julia> bar(0)
0

julia> bar(1)
1

julia> bar(2)
2

Your MWE works with the according changes, but I feel it’s important to point out that the global names fooA and fooB don’t seem necessary, and conditional global names are not good practice in general. What do you need out of the implementation variants exactly?

1 Like
3

Okay, got it, I can use getfield as function to call via @invokelatest.

Regarding the good practice I want to point out, that functions like include_switch are only called once per Julia session. And e.g. :a or :b from the MWE could be

or

That I use different function names depending on the selected file has the advantage, that I can easily see, what is include in the current session (and in what order it is called):

julia> list_everything()
pre_init
--------
demographic__core: set_demographics_rnd_seed (PRIO_FIRST)
population__with_regiostar: initialize_regiostar! (PRIO_NORMAL)
population__with_pt_stop_distances: read_pt_stops (PRIO_NORMAL)
...

reconsider_mob_avail: rational_decision (PRIO_NORMAL)
4

That does make sense from an interactive standpoint. Still, you were working around method table state as part of world age before, and now you have to work around global namespace state as well. You could dodge the latter with a manual global cache of Dict{Symbol, Function}, but that would involve some really custom metaprogramming especially for multimethod functions. At that point, using a module and @invokelatest doesn’t even seem that bad. If you don’t have a dedicated module though, you should; the MWE putting names in Main is very uncomfortable.

You could use RuntimeGeneratedFunctions.jl to throw unique method bodies into a preexisting @generated method definition to dodge world age and manually cache them in a global Dict{Symbol, Any} for retrieval later. But that does come with the @generated limitation of no nested functions (including do blocks and comprehensions) and a hefty amount of refactoring, so this is probably something to know and not do for now.

1 Like