Capture the current definition of a method

How can I capture the current definition of a method in order to be able to invoke it even if it is later redefined ? That is, how to defeat both the standard method redefinition and Revise ?

I’ve looked at the doc, googled it, and searched here for this… it seems I’m missing the right keywords.

1 Like

Like, you want to somehow do this:

julia> f() = 5

julia> f() = 6

julia> @previous_method f()
5

?

2 Likes
julia> f() = 3
f (generic function with 1 method)

julia> Base.get_world_counter()
0x0000000000007a08

julia> f() = 5
f (generic function with 1 method)

julia> Base.invoke_in_world(0x0000000000007a08, f)
3

but why?

2 Likes

Thanks, but it does not ensure the version of the method.

This is that! :grinning:
Why? For “security” reason to ensure I can call the Base method I’m expecting to call. Also to use delegate part of a redefinition to the previous specific definition.

if you need to rely on this to get the correct result, you’re in the wrong place…Also be warned this will be SLOW.

PLEASE DON’T TO THIS. Use different names instead, for example, let func() call _func() where the latter is your “internal” function

3 Likes
  1. Slow: I get this (I’m new to julia but a compiler guy). I don’t get why I’m in the wrong place w.r.t. security ? When the runtime start I can capture some methods, I can be certain to have “first hand” if my module init is in a sysimage. Then I’m protected against any redefinition of the method ? Is there as way to bypass this ? (assuming Base.invoke_in_world itself is protected likewise and excluding low-level stuff like direct llvm/asm generation ?
    (note: this is not a question of correction but of security, as in protection against adversarial behavior of some code loaded after mine. The scare quotes on “security”… hum… because there so much you can do with such a dynamic semantics)

  2. This use case is dual of 1). I don’t get your reply : if I want to override some method of, say, Base.foo but Base.foo is big and does of lot of work and I only need to change a subpath, then I must redefine Base.foo, no ? It is still useful (“safer”) to delegate the larger part of the work to the original Base.foo, and thus inherit any future improvements ?

(I’m from the FP world, but, sometime, only sometime, virtual based inheritance is convenient :sweat_smile:)

(edit: clarity, typo… and again, sorry)

what are you trying to protect against? The methods you “captured” is useless if user don’t also do invoke_in_world when they are not calling your public-facing function. And say they are calling your function, what’s stopping a later package from changing your outer-most functions’ methods?

you understand they could just override your whole module right?

use a internal function to do the un-changing part of your function. Don’t dynamically time-tarvel with invoke, compiler can’t optimize that.

In the end you can do whatever you want, but know that the package produced with this recipe will be hard to integrate into the ecosystem and likely slow (if performance matters)

1 Like

hum… thank for your reply but I still don’t really get it :sweat_smile:

Don’t dynamically time-tarvel with invoke, compiler can’t optimize that.

I get it… written a few compilers in a previous life

use a internal function to do the un-changing part of your function.

it is not unchanging, it is un-changed. They there is an exported, public API method in Base. Let’s call it Base.foo(…). It is 100 lines long. I want to have, say, a custom REPL experience. The REPL do call Base.foo(…). I can override it to customize. But I need only to change a tiny part of Base.foo(…), say a little case depending on some input value. I think it is still better if by override of Base.foo(…) do delegate to the original implementation. (actually I don’t see any other solution but to copy/past the full origin code of Base.foo(…))

if your new foo rely on old foo, just call the old foo_foo. This is a common pattern, where the foo is high level / flexible, _foo is directly doing the fixed set of heavy lifting.

1 Like

you understand they could just override your whole module right?

Yeah… but for scenario 1) I’m in the sysimage. Otherwise I couldn’t do anything meaningful anyway. I don’t care what the code calling my code will do. I just want strongest guarantee that if my code do run, then it does what it was designed to do. E.g. equivalently, all calls from my code could be inlined, full transitive closure downtown Core! (bad idea, but this would be equivalent to what I tried to describe in point 1)

but this is not gonna happen if you use invoke everywhere?.. I’m probably not understanding your complex use-case

if your new foo rely on old foo , just call the old foo_foo . This is a common pattern, where the foo is high level / flexible, _foo is directly doing the fixed set of heavy lifting.

I’m afraid I’m missing something important here…
The old foo is in Base (the actual Base of the Julia distribution).
How can I do this foo_foo ? Really I don’t get it…

which function do you have in mind in Base and what do you want to do with it that you think you need invoke?

Again, there are two very different cases

  1. Say Base.rm or anything like that that could have catastrophic side effects

  2. Say Base.repl_cmd (to continue the example given above)

# My Code
function Base.repl_cmd(cmd, out)
   if cmd.exec == ["foo-bar"]
       # special handling
   else
       orginal_base_repl_cmd(cmd, out)
   end
end

How can I define orginal_base_repl_cmd without invoke_in_world ?

You cannot.

That in and of itself is not a problem, no? What do you want to protect yourself against by forcing an older world age?

1 Like

thank you… So I’m not mad :smiley:
invoke_in_world is indeed what I was looking for… and the only possible way given the fully dynamic nature of the overrides (with back-link and code re-generation).

Question: why is it not in the doc ? I understand that users could abuse it… But the use cas defined in 2) above seems… well useful ?

1 Like

the reason is that usually you want to use the latest methods, everywhere… if somehow X is designed to be customizable but you can’t customize it without doing this hack, then X is poorly designed.

https://github.com/JuliaInterop/Cxx.jl/blob/ce1bbf4a79d348252f07f8a5774e90da75ebbf44/src/CxxREPL/replpane.jl#L243-L252

I mean at least you definitely don’t need invoke for making a new REPL mode…

2 Likes

Because “world age” is an implementation detail to achieve compiled functions in the context of a dynamic programming language with eval.

You don’t have to use invoke_in_world for that.

@eval Base function foo()
     [.. old definition of foo]
     new_var = 5
     do_with_func(new_var)
     [... old definition of foo]
end

Getting the old definition itself can be done by looking up its source code, parsing the code and manipulating the resulting AST (though of course you have no guarantee that the source on disk is the same one that was used to compile the original function in the first place).

That aside, Cassette.jl is one tool doing something like this, but it comes with HUGE compiler and performance drawbacks and on top of that it’s a very brittle tool for all the right reasons.

1 Like

I’ve looked at some of the rewriting tools (Cassette, macro + MacroTools or MLStyle, what was done in the context of the source-to-source autodiff, etc…). It’s very nice.

But it’s heavy for a little thing like…

# My Code
function Base.repl_cmd(cmd, out)
   if cmd.exec == ["foo-bar"]
       # special handling
   else
       orginal_base_repl_cmd(cmd, out)
   end
end

after thinking a tiny bit about it… it is not clear what semantics orginal_base_repl_cmd could have without a global world age. A chain of method re-definition (a “function age” for given fixed symbol) seems tricky with the specialization… Could maybe be defined for monomorphic functions… anyway… thanks for all the info.