Is there a way to get all method definitions in a Module? I tried names(mod;all=true,imported=true) but going this way still doesn’t catch methods extending existing functions using this style: function Base.push!(c::MyContainer,x) etc.
Disclaimer: I do some analysis of Julia code, so I hope this category is appropriate for this question
I was struggling with this myself a few hours back.
Here it is (example using Base):
struct CustomStruct
name::String
end
# dumb method - just for MWE
Base.push!(cs::CustomStruct, x) = CustomStruct(x)
module_names = names(Base, all=false)
functions = filter(x -> isa(eval(x), Function), module_names)
module_methods = map(m -> m => methods(m), eval.(functions)) |> Dict
@info module_methods[push!]
The print will show this:
┌ Info: # 29 methods for generic function "push!" from Base:
│ [1] push!(s::Base.IdSet, x)
│ @ idset.jl:21
│ [2] push!(q::Base.IntrusiveLinkedList{Base.LinkedListItem{T}}, val::T) where T
│ @ linked_list.jl:138
│ [3] push!(q::Base.IntrusiveLinkedList{T}, val::T) where T
│ @ linked_list.jl:52
│ [4] push!(c::AbstractChannel, v)
│ @ channels.jl:10
│ [5] push!(W::Base.IntrusiveLinkedListSynchronized{T}, t::T) where T
│ @ task.jl:698
│ [6] push!(node::SN, child::SN) where SN<:(JuliaSyntax.TreeNode{<:JuliaSyntax.AbstractSyntaxData})
│ @ JuliaSyntax ~/.julia/packages/JuliaSyntax/irSvg/src/syntax_tree.jl:202
│ [7] push!(s::Set, x)
│ @ set.jl:103
│ [8] push!(cs::CustomStruct, x)
│ @ Main ~/code/UniLM.jl/src/macros.jl:87
... and so on
You can see the method for my CustomStruct here.
P.S. Ignore that I was working at line 87 in a file named macros.jl 
@algunion thank you! I think it doesn’t quite cut it.
@info module_methods[push!]
This assumes you know which function you’re interested in (push! in this example). I don’t know that. I only have a Module object. And using only it as the input, I want to get all method definitions inside it.
Generally, you approach is similar to what I’ve been doing so far, and, as I said above, I don’t see a way to catch method definitions for functions created elsewhere (e.g. Base.push!) if you don’t know in advance which functions were extended in this module.
I suggest to run the code and see for yourself - the module_methods contains all the methods defined for the functions.
The only reason I singled out push! is that you mentioned the following:
So I wanted to show that my example includes even the push! the method signature defined on my custom struct.
You can ignore that last line where I show the methods for push! - that was just to show that [8] push!(cs::CustomStruct, x) is there.
My code doesn’t assume anything about the push! - you can replace Base with your Module and the module_methods will result in a Dict mapping function => methods for all the functions in your Module.
I explained above why I wanted to show you the push! methods.
Your code misses methods as I described above. Here’s an example:
shell> cat x.jl
all_methods(mod::Module) = begin
eval(sym) = Core.eval(mod, sym)
module_names = names(mod, all=true)
functions = filter(x -> isa(eval(x), Function), module_names)
module_methods = map(m -> m => methods(m), eval.(functions)) |> Dict
end
julia> include("x.jl")
julia> module M; Base.push!()=1; end;
julia> all_methods(M)
Dict{Function, Base.MethodList} with 2 entries:
eval => # 1 method for generic function "eval":…
include => # 2 methods for generic function "include":…
So, no Base.push!, only eval and include.
I get it now - you want to get all the methods from all the modules that are accessible from inside your Module.
You just need to run it in a recursive way - get the names from your module (Base will be included) and then call all_methods on all modules that show up in your names(Module).
No, I don’t want all methods of all accessible modules. Only methods defined in this module. The example module M above defines exactly one method. And I want a way to get exactly that (I don’t mind couple extra special methods like eval and include but not “all methods of all accessible modules”!) .
Also, it’s not clear to me how you mean to get modules on which the given module depend. The names function doesn’t give you that info.
Clear - give me 5 minutes.
I don’t know a way to do this, but here’s a small example of how names doesn’t work:
julia> module Example
export f
import Base: +, -
f() = 1
Base.identity(::typeof(f)) = 1
+(::typeof(f), ::typeof(f)) = 2
end
Main.Example
julia> names(Example)
2-element Vector{Symbol}:
:Example
:f
julia> names(Example; all = true)
7-element Vector{Symbol}:
Symbol("#eval")
Symbol("#f")
Symbol("#include")
:Example
:eval
:f
:include
julia> names(Example; imported = true)
4-element Vector{Symbol}:
:+
:-
:Example
:f
None show the module’s method definitions with signatures, just names, and not even the right names. identity never shows up, but - did.
# correct definition names
5-element Vector{Symbol}:
:+
:eval
:f
:identity
:include
You can at least inspect method definitions given the names and ignore the names with no definitions. But since :identity is missing, we don’t see that definition.
julia> [methods(getproperty(Example, name), Example) for name in names(Example; all = true, imported = true)]
9-element Vector{Base.MethodList}:
# 0 methods for type constructor
# 0 methods for type constructor
# 0 methods for type constructor
# 1 method for generic function "+":
[1] +(::typeof(Main.Example.f), ::typeof(Main.Example.f)) in Main.Example at REPL[1]:6
# 0 methods for generic function "-"
# 0 methods for callable object
# 1 method for generic function "eval":
[1] eval(x) in Main.Example at REPL[1]:1
# 1 method for generic function "f":
[1] f() in Main.Example at REPL[1]:3
# 2 methods for generic function "include":
[1] include(mapexpr::Function, x) in Main.Example at REPL[1]:1
[2] include(x) in Main.Example at REPL[1]:1
Here you have it.
Pretty messy code - but it works - you can make it beautiful.
get_methods(f, definedin::Module) = filter(mtd -> mtd.module == definedin, methods(f))
all_methods(parent::Module, target::Module; rec=true) = begin
module_names = names(target, all=false)
functions = filter(x -> isa(eval(x), Function), module_names)
mm = Dict()
if rec
modules = filter(x -> isa(eval(x), Module), module_names)
for m in modules
mm[m] = all_methods(parent, eval(m), rec=false)[m]
end
end
parent_methods = filter(kv -> !isempty(kv[2]), map(f -> f => get_methods(f, parent), eval.(functions)))
mm[Symbol(target)] = Dict(parent_methods)
return mm
end
It will only return methods that are defined inside parent. So, initially you can call all_methods(M,M).
Just test it on your end and let me know.
P.S. I removed the eval(sym)... from the function. I suggest defining that somewhere else (because it the function is recursive now).
Some additional context: it was the same approach as I initially proposed. But there is a field module available for Method. So I just used that to test if the method was defined in your module (parent in this context).
This is a big chunk of code - I would suggest breaking it into multiple functions to clarify things.
@Benny, identity is part of Base - because you defined it in that way.
However you can do names(Base) and you’ll find it there.
Now, you can do methods(identity) and extract the one defined in Example by checking method.module == Example.
which can identify the method definition as occurring in Example:
julia> parentmodule(identity)
Base
julia> @which identity(1)
identity(x) in Base at operators.jl:526
julia> @which identity(Example.f)
identity(::typeof(Main.Example.f)) in Main.Example at REPL[1]:4
I believe ulysses wants to find this information from Example alone.
Let’s see after he tries my latest attempt.
Well I can try it for my Example before then
julia> all_methods(Example,Example)
ERROR: UndefVarError: f not defined
Stacktrace:
[1] top-level scope
@ :0
[2] eval
@ ./boot.jl:368 [inlined]
[3] eval
@ ./client.jl:478 [inlined]
[4] #4
@ ./REPL[2]:3 [inlined]
[5] filter(f::var"#4#8", a::Vector{Symbol})
@ Base ./array.jl:2539
[6] all_methods(parent::Module, target::Module; rec::Bool)
@ Main ./REPL[2]:3
[7] all_methods(parent::Module, target::Module)
@ Main ./REPL[2]:1
[8] top-level scope
I believe this is the issue:
julia> names(Example, all=false)
2-element Vector{Symbol}:
:Example
:f
julia> eval(:f)
ERROR: UndefVarError: f not defined
Maybe try getproperty(target, x) instead of eval(x)? You could narrow it down to the module with eval(target, x), but I’d rather access something I know is already there instead of evaluating an expression.
Fixing now - I tested from the same module where all_methods was defined.
I don’t want to do names(Base) – it defines a lot of stuff that I have no interest in.
Your recursive solution doesn’t work either: it traverses only submodules of a given module; e.g. when you do module M; module N; ... end; #= N =# end; # M) it will find N, but it will not find modules M makes use of (like Base).
I understand.
For now - the only feasible way I can see it working is this:
Base - because when you do Base.push!(...) = ... you only associate the new method with your caller module, but the function name is still defined in Base - and to get the methods for your function, you actually need that function name - and that is part of Base).method.module indicating in which module the method was defined)However, since you clearly stated that you want to avoid something like names(Base), I don’t think there is value in fixing the function above to work from outside modules.
If you still want to give it a try - you can execute it from inside the module of interest. It should work.
I wish you luck in finding a solution that satisfies the restriction related to avoiding names(Base).
An alternative way to perform your analysis is to parse the code and inspect the Expr objects.
This will not cover cases where you had some dynamically generated functions at runtime - but if you are interested only in the source code, the parsing method + Expr analysis will do the job.