I maintain a VS Code–based editor that bundles a fork of julia-vscode. We currently don’t start LanguageServer.jl, partly to avoid SymbolServer.jl’s large serialized symbol cache and a fragile re-indexing path. We’re also evaluating JETLS.jl as a possible long-term replacement, so its indexing and cache behaviour matter to us. A few questions:
-
How does JETLS provide completions, hover information, and documentation for packages that are present in the active project but have not yet been loaded/imported by the code being analysed? Does it proactively analyse the project environment/dependencies, or only code that becomes reachable from files being analysed?
-
Does JETLS maintain any persistent on-disk indexes or metadata of its own (similar in spirit to SymbolServer’s serialized symbol store)? If so, what information is stored there?
-
As additional packages are installed over time, should we expect any significant JETLS-specific cache growth, or primarily the normal growth of ~/.Julia/compiled and other Julia-managed caches?
-
Is there any rough guidance on expected disk footprint and first-analysis/indexing cost for a large project environment?
The reason I’m asking is that reusing Julia’s existing compilation/precompilation infrastructure, rather than maintaining a second symbol database that needs to be kept in sync, would be a significant advantage for our use case. Thanks!
@aviatesk can answer these questions better than me, but from what I understand:
- JETLS only loads packages that are actually used by your project
- No, it loads every dependency into a Julia process, but does not store anything on disk
- No
- No additional disk footprint, large initial analysis cost
Can you elaborate on this? The symbol server cache files are typically very small (megabytes, at most); this can of course add up over time, but shouldn’t really be prohibitive in most environments. In my experience, re-indexing is the biggest problem, since loading packages into a Julia process is pretty expensive – we should get decent coverage for cloud-indexed packages at some point soonish though.
What @pfitzseb understands about JETLS is correct.
To add a bit more on point 4, the disk footprint is as @pfitzseb described. For memory usage, I think it makes sense to expect something roughly comparable or larger to loading the packages being analyzed into a Julia process.
Depending on the project, the first analysis can take quite a while.
To give you a rough idea, on my machine, analyzing JETLS itself takes about 30 seconds with 6 threads. When I analyzed CSV.jl recently, it took about 20 seconds.
Also note that the current incremental analysis implementation is still fairly rough, so there is still a noticeable cost on every save. This is expected to be improved in the near future.
Thanks both. This is exactly the clarity I needed.
Fair pushback: “large serialised symbol cache” was the wrong characterisation on my part, and I should walk it back. You’re right that the symbol store itself is modest.
But it sent me to actually measure it, and the picture is interesting. That “Cache 1.32 GB” in the extension’s details panel isn’t the extension code (the “Size”, 30 MB); it’s VS Code’s tally of what the extension has written to its globalStorage. The breakdown on that install:
symbolstorev5 is 255 MB. SymbolServer’s serialized symbol cache (one file per package/version I assume, so it accumulates across everything we have indexed)lsdepot is 1.1 GB. A separate, private Julia depot the extension maintains to run LanguageServer.jl, mostly precompiled packages
So the symbol store is the smaller part. The real footprint is that private LS depot. And it’s per-editor. Each VS Code-family app gets its own globalStorage. Tellingly, our own fork, which doesn’t start LanguageServer.jl, sits at 193 MB, versus ~1 GB+ for the editors that do.
So my concern was misnamed: less “the symbol cache”, more “a second, per-install precompiled depot to maintain”. Which is exactly why, if I understood correctly, JETLS reusing Julia’s normal compiled cache, rather than a private depot, is appealing for a fork we would like to distribute.
I think we are probably going to move away from the private Julia depot for the LS, so that problem might go away at some point.
