Parsing Julia code to separate symbols based on scoping and create a symbol table

General Usage
,
1

I have a quote block in Julia containing functions, modules, and etc. Assuming that quote block is in Main module and Main module does not have these symbols with same names. I want to separate the symbols based on their scoping and create a symbol table that stores their names which reflect their scopes along with their locations in AST where they occur. Specifically, I want to handle cases where symbols with the same name have different scoping.

exp = Meta.parse("quote
    x = 2
    
   function foo(x)
        y = 2 * x
        z = x + y
    end
    
    function bar(a)
        y = a + x
    end
end")

In this quote block, there are multiple symbols named x and y, … each with its own scoping. I want to parse this code and generate a symbol table that reflects this scoping, something like:

Symbol: x
 ScopeName: x@_1, Location: exp.args[1].args[2].args[1] (or) (1,2,1)
 ScopeName: x@_2, Location: exp.args[1].args[4].args[1].args[2] (or) (1,4,1,2)
 ScopeName: x@_2, Location: exp.args[1].args[4].args[2].args[3].args[2].args[3] (or) (1,4,2,3,2,3)
 ScopeName: x@_2, Location: exp.args[1].args[4].args[2].args[5].args[2].args[2] (or) (1,4,2,5,2,2)
 ScopeName: x@_3, Location: exp.args[1].args[6].args[2].args[3].args[2].args[3] (or) (1,6,2,3,2,3)
Symbol: y
 ScopeName: y@_1, Location: exp.args[1].args[4].args[2].args[3].args[1] (or) (1,4,2,3)
 ScopeName: y@_1, Location: exp.args[1].args[4].args[2].args[5].args[2].args[3] (or) (1,4,2,5,2,3)
 ScopeName: y@_2, Location: exp.args[1].args[6].args[2].args[3].args[1] (or) (1,6,2,3,1)
.
.
.

Could someone provide guidance on how to achieve this in Julia? Additionally, are there any existing packages or libraries that could simplify this task?

1 Like
2

I had the same question on slack some time ago (February, so the slack thread will disappear sometime soon):

What I understood from @vchuravy and @Mason 's guidance was that this is not currently possible in an easy way.

The parsing of julia code was recently moved from lisp to actual Julia code thanks to @c42f 's JuliaSyntax.jl. However, you need to introspect the lowering step to achieve what you want, which happens after parsing. That still happens in lisp, so we can not introspect it. Claire is also working on GitHub - c42f/JuliaLowering.jl: Julia code lowering with precise provenance which might help, but that is still at a very embryonic stage.

Of course, you can write some AST crawers yourself and do all of this yourself, but then you probably miss some edge cases.

4 Likes