IRTools/Cassette question: Is it possible to replace everything with invokelatest?

General Usage
1

Hi there,
I would like to build a rewriting system which ensures that everything will call the latest methods. This could be useful for developing long running applications like web servers.

goal

allinvokelatest() do
  webserver()
end

This should replace it to invokelatest(webserver), as well as changing the latest webserver definition (lowered code) to also call invokelatest everywhere such that their latest definition also gets rewritten recursively.

Is this possible?

2

Why does this need to be recursive? If you call invokelatest(webserver), any code called as part of webserver should have access to the latest methods up to that point, so there’s no need for a recursive transformation.

The only issue I could imagine would be if webserver itself defined new methods, but even in that case replacing any call with invokelatest seems like overkill, since the transformation does not need to be recursive beyond that point.

2 Likes
3

This is unfortunately not true. By now I understood why:

if you do invokelatest(webserver), the maximum world age will be fixed to the current world age.
If you then adapt some code, a newer world age is created, however everything within the invokelatest won’t see it, because its maximum is now lower.

Unfortunately, even Base.invoke_in_world does not work, because it ignores worlds which are larger than the current world.

It would be great if we somehow could define our own invokelatest, but change the current world to typemax(UInt)

JL_CALLABLE(jl_f__call_latest)
{
    jl_task_t *ct = jl_current_task;
    size_t last_age = ct->world_age;
    if (!ct->ptls->in_pure_callback)
        # change this to `typemax(UInt)`
        ct->world_age = jl_atomic_load_acquire(&jl_world_counter);
    jl_value_t *ret = jl_apply(args, nargs);
    ct->world_age = last_age;
    return ret;
}

but this is C code - how can we write something like this in Julia?

4

Ah, so webserver spawns some background task and you’re defining new methods while that task is still running? What kind of methods are supposed to be redefined? For example, if you are listening for http requests and you have a user-defined callback that may be redefined, you don’t need invokelatest everywhere, you just invoke the callback with invokelatest.

Messing with internals here is unlikely to be very fruitful, since the current semantics are are not without reason. Otherwise, the compiler might get into an inconsistent state resulting in miscompilation.

1 Like
5

thank you for the feedback. Okay, so no typemax(UInt) ? I guess, I will just open a suggestion on Julia github to get further feedback here the issue

Meanwhile I build the IRTools implementation. It has some difficulties with distributed/@async, threads and such (all of which should just be special cases of rewrites), but otherwise it works :slight_smile: