SymbolServer, CSTParser and JuliaSyntax

Tooling VS Code
1

I am trying to understand the relationship between the packages used by LanguageServer.jl, which powers Julia VSCode.

julia-vscode
julia-vscode601×373 12.8 KB

Question:

  • which feature/ requirement is implemented by CSTParser/JuliaSyntax
  • which feature/ requirement is implemented by SymbolServer.jl ?

Key features/ requirements:

  1. show the function documentation when you hover with the mouse of the name of a function call
  2. implement “go to definition” using the right mouse click
  3. display warnings if there is a syntax error in a line of code

Is my understanding correct that for 1. and 2. only SymbolServer.jl is needed, and for 3. CSTParser/JuliaSyntax is needed?

2

CSTParser/JuliaSyntax are used to parse Julia source code and create a node tree for the syntax elements, and they are the source of syntax error diagnostics.

We can extract a lot of information that way from code, but there are limits: if there is a package that for example uses @eval to generate functions, we won’t know about those functions by just looking at the syntax (and there are lots of other similar things). So, we have a second strategy to learn about the stuff in packages that are part of a user project: we load the project into a running Julia session and then use runtime inspection to learn about the functions/methods/docstrings etc that are defined in that package. The benefit of this second strategy is that it is pretty complete, i.e. we get pretty much all the info we need. The downside is that it is slow and certainly (at the moment) impractical to run on each key stroke. So, we only use this strategy for packages that are not in the current workspace but are referenced by the active project (roughly). The functionality for extracting that info is in SymbolServer. We also cache that info on disc and in the cloud for faster access, also all handled by SymbolServer.

StaticLint is then the package that essentially takes the info from the parsing and from SymbolServer, puts it all together and tries to come up with all the semantic information that is needed for various features, plus all the lint rules etc.

I think for 1 and 2 probably everything is needed. For 3 it is just the parsing packages.

2 Likes
3

Thanks for the detailed explanation.

But perhaps it would better to have something simpler that is more reliable. For example, I don’t care so much about statements that are executed with “eval”. But I do care about 1. and 2. working for tests and examples in a package that is checked out from Github.

4

So the situation right now is (I think) that the info that we get from SymbolServer is very reliable and good, but things are less smooth for stuff that we try to extract directly from the syntax trees.

There is no way that we can move entirely to the SymbolServer model of things, that would never work for code that one edits.

Given that, I don’t really think it makes sense to cut stuff out, I think the right move is to improve the parts that rely on the syntax trees as much as we can, and leave the (well functioning) SymbolServer stuff in place.

1 Like
5

Also, I should say, I think another problem is that actually we just don’t handle certain project/test project scenarios properly, but none of that would be any easier if we cut anything out, we just need to handle those scenarios…

1 Like