Side Effect Analysis on Macros

Tooling VS Code
1

A recurrent challenge for linting Julia is macro mini languages. From chaining with underscores to sum types, there are lots of symbols that get created by macros which are obvious to the human reader but not to LSP. The obvious solution would be to expand macros before analyzing the AST, but since macros can execute arbitrary code, that could be dangerous.

Given that we have side effect analysis for functions now, could that be leveraged to determine whether expanding a macro would do anything other than return AST and only expand macros that are pure? I don’t know if the compiler applies the same effect analysis to macros.

2

Pluto seems to do some macro analysis/expansion, wonder how specifically they do it…

1 Like
3

Pluto has access to a Julia runtime that has the macro and everything it depends on loaded, which is not the case for the VS Code extension.

Generally full static analysis of macros is basically impossible, so an idea is to move to a multi-process approach, where some sandboxed processes are in fact allowed to load packages and expand macros in a module context.

2 Likes
4

One alternative, (I’m not a fan, but it might work alright) is to have a plugin system so that macro-writers can communicate to the linter what variable names it is creating, and what types it is defining, and other static information the linter might need to know.

1 Like
5

Yes, but it’s also not entirely clear how to best design this plugin system. Either it also requires loading LS-extension-code based on your current environment or is pretty limited in that it can only work declaratively.

6

I know I’ve said it elsewhere to you, @pfitzseb , but could you clarify if there is any possibility of convincing LSP that certain symbols are definitely defined? Like in a list in a Workspace config file? That way, one could white list, say, _ if they are using Chain.jl.

7

Okay, one more thought, is it possible to use comments to communicate with the language server? Like, could I put #=expand=# @chain or something when I’m very confident that I want it expanded? Then package documentation could add notes to their macros about which are safe to notate as such.

8

I recently found out that Clojure’s clj-kondo can be extended with hooks written in Clojure (only interpreted, not compiled). I don’t really know how powerful this feature is, but it’s some prior art you could look into.

i like that idea because if you want to grow a language, then you need tooling that grows with it.

1 Like
9

A hacky but very useful trick is to use local var before macros that result in new variables etc. For example, all of these annoying warnings:

image
image716×401 40.1 KB

can be silenced with local expressions like this:

using SumTypes

local Either, A, B, Left, Right
@sum_type Either{A, B} begin
    Left{A}(::A)
    Right{B}(::B)
end

function foo() :: Either{Int, Float64}
    # Randomly return either a Left(1) or a Right(2.0)
    rand(Bool) ? Left(1) : Right(2.0)
end

using Symbolics

local x
@variables x

function f()
    local y
    @variables y
    return x + y
end
2 Likes
10

Hi!

Is there any update here? It turns out that the LSP is almost unusable if you use Paramters.jl to define all the custom structures in your package.

11

Switching my workflow to use more Pluto is about all the progress I can report unfortunately!

1 Like