Get all names accessible from a Module

General Usage
1

I know that Base.names can be used to extract most of the names that are accessible from inside a Module m, but I just realized that even if one calls it names(m;all=true, imported=true), the list of returned names does not include the ones that are exposed inside m because of using other packages.

As an example suppose you have the following simple module definition

module TESTMODULE
using BenchmarkTools
end

calling names(TESTMODULE;all=true, imported=true) does not return any of the names exported by BenchmarkTools.

Is there a way to extract all of the variables/functions that are accessible from TESTMODULE above?

2 Likes
2

no one?

help?> names

  names(x::Module; all::Bool = false, imported::Bool = false)

  Get an array of the names exported by a Module, excluding deprecated names. If all is true, then the list also includes non-exported names defined in the
  module, deprecated names, and compiler-generated names. If imported is true, then names explicitly imported from other modules are also included.

Well, with imported=true you get not all “names explicitly imported …” as I would normally read the docstring. But at least all imported/used modules are listed - thus you can then call

names(BenchmarkTools; all=false)

to get all names exported from BenchmarkTools and therefore imported into TESTMODULE (see also code below).

But what if we have a non-exported identifier explicitly imported?

module M0
f1(x) = x
f2() = "f2 available"
export f1
end

module M1
import ..M0
using ..M0: f2

nms = names(@__MODULE__, all=true, imported=true)
mods = [n for n in nms if getproperty(@__MODULE__, n) isa Module]
imported = vcat([names(getproperty(@__MODULE__, mod), all=false) for mod in mods]...)
allnames = union(nms, imported)
println(f2())
@show nms
@show imported
@show allnames
end

julia> 
f2 available
nms = [Symbol("#eval"), Symbol("#include"), :M0, :M1, :eval, :include]
imported = [:M0, :f1, :M1]
allnames = [Symbol("#eval"), Symbol("#include"), :M0, :M1, :eval, :include, :f1]

Any idea how to get f2 listed?

Another question: the names() docstring doesn’t IMO reflect the behavior of the function. What is the intended behavior then?

2 Likes
3

How about this?

module M1
    import ..M0: f2
end

names(M1, imported=true)

image
image1326Ă—156 16.4 KB

1 Like
4

Thank you klwlevy, that worked. The difference between your code and mine is that I have the line

nms = names(@__MODULE__, all=true, imported=true)

within the module, and you call names(M1, imported=true) after the execution of M1.

Unfortunately your suggestion doesn’t solve my problem, as I need the list of available identifiers “at the point of use”- within the module M1, and I do get all of them - except those explicitly imported.

5

@Eben60
I think that here the issue is not whether you call the names function after module execution.

The example provided by @klwlevy works because he did import ..M0: f2 rather than using ..M0: f2.

If you switch to using import it will also work and appear under the nms variable in your code.

The problem still remains that if variables are imported with using (explicitly or not) they can not be extracted with names

2 Likes
6

disberd, thank you, you are correct on both points:

  • That it is import vs using , not within vs. after
  • and that doesn’t solve my problem.

It is even worse than I thought: The modules made available through using are completely “invisible” for the names function

module M4
using ..M0
end
println(names(M4, all=true, imported=true))

julia> 

[Symbol("#eval"), Symbol("#include"), :M4, :eval, :include]

Any ideas?

1 Like
7

I probably don’t understand what your problem is.

Please state if this solves your problem or, alternatively, what you are lacking.
It makes use of the package Reexport.jl.

module M0
    f1(x) = x
    f2() = "f2 available"
    f3(x) = sin(x)
    export f1, f3
end

module M1
    # This is a separate package
    using Reexport
    # Re-exporting exported symbols from M0 also in M1
    @reexport using ..M0
    # Re-exporting the non-exported symbol f2 from M0 in M1
    @reexport using ..M0: f2
end

M1.f1(42)
M1.f2()
M1.f3(3.14/2)
names(M1)

names(M1) = [:M0, :M1, :f1, :f2, :f3]
8

Thank you @klwlevy, unfortunately that doesn’t solve my problem, which is

I need the list of available identifiers “at the point of use”- within the module M1

without otherwise changing the code of the module (e.g. switching form using to import or @reexport), as that code is not mine.

This has to do with this topic : Upon wrapping a struct into a module through a macro I need to make the original context, or selected parts of it, available within this module. The problem is to obtain all names in the original context.

9

It’s weird:

# freshly started Julia
using Unitful

module M0
using Unitful
end

println(names(M0, all=true, imported=true))
println(names(Main, all=true, imported=true))

julia> 
[Symbol("#eval"), Symbol("#include"), :M0, :eval, :include]
[Symbol("##meta#58"), Symbol("#1#2"), Symbol("#3#5"), Symbol("#4#6"), :Base, :Core, :InteractiveUtils, :M0, :Main, :Revise, :Unitful, :VSCodeServer, :eval, :include]

For the Main module, names() returns Unitful , but for M0 - not. I doubt this is intended behavior. Anybody knows whom I could ask on that point?

10

@Eben60 I think you would need to follow a different approach.

From the post you linked I guess your use case is recreating the struct defined at a certain point within the module by creating a temporary submodule where you must re-import all the variables from the calling module that are used in the struct definition.

I have to do something similar to extract variables from Pluto in PlutoVSCodeDebugger.

I have a very simple function there that collect variables names that are being found in an expression only if they are defined in a targe module (in your case it will be the one where your custom macro is called from) and they are not defined neither in Main nor Base (the purpose for Base is to avoid names that are automatically re-imported from Base in every module, excluding also Main was just because in my case the generated expression is evaluated in Main, so it’s pointless to also track things already defined in Main).

Here is the function:

And it puts the found variable names inside a Set of Symbols (I use a Set so that I automatically avoid duplicates).

While it may not be already applicable as is to your use case, I think it can be a good starting point for experimenting.

EDIT:
Here is a small example usage

julia> _names = Set{Symbol}()
Set{Symbol}()

julia> module M
           a = 1
           b = 2
       end
Main.M

julia> PlutoVSCodeDebugger.extract_variables!(:(@kwdef struct Foo
               x = a
               y = b
           end); _mod = M, _names)

julia> _names
Set{Symbol} with 2 elements:
  :a
  :b

You could use the extracted names to specifically re-import from the calling module with an expressione like the following

julia> Expr(:import, Expr(:(:), Expr(:., :., :., nameof(M)), map(x -> Expr(:., x), collect(_names))...))
:(import ..M: a, b)

I also do something similar to extract variable names to re-export for the @fromparent macro from my other package GitHub - disberd/PlutoDevMacros.jl: Simple macros to help developing packages using Pluto.jl as base (which is also the reason why I opened this thread in the first place).
So I had my share of headaches with macros and I might have stumbled on similar problems to what you will face :slight_smile:

2 Likes
11

@disberd, thanks a lot. So if I understand you correctly, you are suggesting to go bottom-up and collect all identifiers recursively from the expression to be processed by my macro. Yes, that can be a viable approach, I’ll try it as soon as I have some spare time. One question now: What would happen if there is a macro within that expression?

Still I think it’s worth open an issue for the names function - would do it later on, too. It’s annoying it doesn’t return all identifiers in the given scope.