VScode Julia server fails precompilation

Running julia --pkgimages=no seems to have gotten around the initial issues. Gravlax is using macOS Mojave.

This seems to be another case of Can't install packages on v1.9 MacOS · Issue #49775 · JuliaLang/julia · GitHub

What would the advantage be to that over the programmer putting in a try/catch into the @compile_workload block? In general I’m a little reluctant to hide errors unless there’s a really good reason to do so. For example, open(f, filename) definitely needs a try catch around f in order to satisfy its promise to close the file after calling f, but there’s no guarantee I know of that precompilation should succeed even for buggy workloads. (I’m sure you know that’s not an accusation about the quality of your LanguageServer workload. If I had a nickle for every buggy workload I’ve written…)

If try/catch is needed, I’d rather it be explicit in the code that everyone can see when inspecting the package.


There are two separate issues. One is logging. The second is error handling.

The logging issue belongs to other layers such as the individual package or Pkg which orchestrates parallel precompilation. I’ll defer further discussion of that. The main helpful thing PrecompileTools.jl could do here is perhaps note thst the error occurred while using its macros since macros are not a very clear part of a stack trace.

The other issue is error handling.

Packages should not fail during precompilation or loading, especially if the failure is not critical to the function a package. The consequences for failing are severe since all packages which depend on the failing package will also fail to load. A package which does not load at all is also much more difficult to debug.

The work being done during PrecompileTools.jl’s @setup_workload and @compile_workload is not critical to module loading. This is only run when generating cached code to disk. If the workload had to run, it should not occur within those macros. Module compilation can be turned off by command line arguments to Julia. Thus, anything run within these macros is only for optimization. A failure during an optimization step should not cause a package to fail to load. An unexpected error here should still be reported, but not cause precompilation and module loading to fail. Because these macros should not fail, PrecompileTools.jl should help ensure that.

In the case of LanguageServer.jl, the current @compile_workload always results in an error being reported. That specific error should be suppressed.

Catching errors during @compile_workload, reporting them, and resolving them may require repetitive non-trivial code.

  1. PrecompileTools.jl may be better situated to address certain errors. Issues with Preferences.jl and global options come to mind. Another situation may be insufficient disk space.
  2. Reporting these errors should be well coordinated with the logging system, and we should build facilities to re-report these errors at a different time. For example, one may want to issue a warning that @compile_workload precompilation was not successful on subsequent attempts to load the module.
  3. Resolving these issues require a specialized facility to invalidate the cache and force precompilation again. Some of this might be a force keyword to Pkg.precompile. However, PrecompileTools.jl might be able to uniquely detect non-critical @compile_workload errors, and subsequently encourage those packages to be precompiled again.

What I’m discussing here may be getting confused with another issue that is particular to LanguageServer.jl. I’m not trying to hide errors. Rather, I would like to

  1. Make the @compile_workload errors non-fatal to package loading and precompilation by default.
  2. Improve reporting of those errors and providing the ability to re-report them at a later time when parallel precompilation is not happening.

This error has nothing to do with the new precompile supported added to LanguageServer. The original error message shows that this is from the extension version v1.38, which was released in October, long bevor anything related to precompile was merged.

@Gravlax can you update your Julia extension to the latest version and see whether you still see that error?

1 Like

Well said. I can’t reproduce this issue in an updated plugin

yes, thanks. I did downgrade the LanguageServer just to see if it could solve the issue.
That’s what I get with the latest version:

I think the error here is a precompilation problem due Julia 1.9 pkgimages. The pkgimages cannot created due to lld missing some depedency on an old macOS.

@Gravlax , does Julia 1.8 work fine?

@davidanthoff , is there is an easy way to pass --pkgimages=no to the Julia instance running the LanguageServer?

yes, 1.8 (and previous ones) works like a charm.

No, we tightly control how that instances is started :slight_smile:

I think the error here is a precompilation problem due Julia 1.9 pkgimages.

That would suggest that precompile should also not work outside of VS Code, right?

Right, in my case, yes.

I just had opportunity to opportunity to work on this machine again. I have tested Julia 1.10 with latest VScode and Julia language support v1.54.2. Unfortunately the problem persists.
There is an issue with write permissions:

The Language Server failed to precompile.
Please make sure you have permissions to write to the LS depot path at
	/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1

However, write permissions do not seem to be the issue: I added write permissions to all and there was no effect.
In fact I wonder if the space in Application Support could the issue. On this OS, this path is simply not valid: /Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1
Only this one is valid:
/Users/imac/Library/Application\ Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1
Is there a way to manually edit this path? Adding the backslash may solve this issue and probably that one too.

Can you please share the output of versioninfo()? What version of macOS are you using? Also, can you pleas paste (no screenshots) the full error message you’re getting?

thanks, here’s versioninfo()

julia> versioninfo()
Julia Version 1.10.0-beta3
Commit 404750f8586 (2023-10-03 12:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 4 × Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, ivybridge)
  Threads: 2 on 4 virtual cores

and the complete error message is:

Activating project at `~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/environments/languageserver/v1.10`
[ Info: Starting the Julia Language Server
ERROR: LoadError: failed process: Process(setenv(`/Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld -flavor darwin -arch x86_64 -undefined dynamic_lookup -platform_version macos 13.3.1 13.3 '' -dylib -o '/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1/compiled/v1.10/JSON/jl_rTIxey' -all_load '/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1/compiled/v1.10/JSON/jl_007D3J' '' -L/Applications/Julia-1.10.app/Contents/Resources/julia/lib -L/Applications/Julia-1.10.app/Contents/Resources/julia/lib/julia -L/Applications/Julia-1.10.app/Contents/Resources/julia/lib -ljulia -ljulia-internal`,["PATH=/Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia:/Applications/Julia-1.9.app/Contents/Resources/julia/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin", "DYLD_FALLBACK_LIBRARY_PATH=/Applications/Julia-1.10.app/Contents/Resources/julia/lib/julia:/Applications/Julia-1.10.app/Contents/Resources/julia/lib:/Users/imac/lib:/usr/local/lib:/lib:/usr/lib", "__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x2", "OPENBLAS_NUM_THREADS=1", "JULIA_LANGUAGESERVER=1", "JULIA_LOAD_PATH=/Users/imac/.vscode/extensions/julialang.language-julia-1.54.2/scripts/environments/languageserver/v1.10/Project.toml:/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1/environments/v1.10/Project.toml:/Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10", "OPENBLAS_DEFAULT_NUM_THREADS=1", "HOME=/Users/imac", "JULIA_NUM_THREADS=1", "JULIA_DEPOT_PATH=/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1", "OPENBLAS_MAIN_FREE=1"]), ProcessSignaled(6)) [0]

  [1] pipeline_error
    @ Base ./process.jl:565 [inlined]
  [2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
    @ Base ./process.jl:480
  [3] run
    @ Base ./process.jl:477 [inlined]
  [4] link_image (repeats 2 times)
    @ Base.Linking ./linking.jl:166 [inlined]
  [5] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2373
  [6] compilecache
    @ Base ./loading.jl:2334 [inlined]
  [7] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1968
  [8] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:92
  [9] #mkpidlock#6
    @ FileWatching.Pidfile /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:87 [inlined]
 [10] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile /Applications/Julia-1.10.app/Contents/Resources/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:110
 [11] #invokelatest#2
    @ Base ./essentials.jl:889 [inlined]
 [12] invokelatest
    @ Base ./essentials.jl:884 [inlined]
 [13] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2980
 [14] maybe_cachefile_lock
    @ Base ./loading.jl:2977 [inlined]
 [15] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1964
 [16] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1806
 [17] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [18] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [19] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1797
 [20] macro expansion
    @ Base ./loading.jl:1784 [inlined]
 [21] macro expansion
    @ Base ./lock.jl:267 [inlined]
 [22] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
 [23] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [24] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [25] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1740
 [26] include
    @ Base ./Base.jl:489 [inlined]
 [27] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2216
 [28] top-level scope
    @ stdin:3
in expression starting at /Users/imac/.vscode/extensions/julialang.language-julia-1.54.2/scripts/packages/LanguageServer/src/LanguageServer.jl:1
in expression starting at stdin:3

The Language Server failed to precompile.
Please make sure you have permissions to write to the LS depot path at
	/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1

┌ Error: Some Julia code in the VS Code extension crashed
└ @ Main ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/error_handler.jl:15
ERROR: Failed to precompile LanguageServer [2b0e0bc5-e4fd-59b4-8912-456d1b03d8d7] to "/Users/imac/Library/Application Support/Code/User/globalStorage/julialang.language-julia/lsdepot/v1/compiled/v1.10/LanguageServer/jl_4Fv5BA".
 [1] top-level scope
   @ ~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/languageserver/main.jl:77
 [2] include(mod::Module, _path::String)
   @ Base ./Base.jl:489
 [3] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:318
 [4] _start()
   @ Base ./client.jl:552
[Error - 12:39:18] Server initialization failed.
  Message: Pending response rejected since connection got disposed
  Code: -32097 
[Info  - 12:39:19] Connection to server got closed. Server will restart.
[Error - 12:39:19] Julia Language Server client: couldn't create connection to server.
  Message: Pending response rejected since connection got disposed
  Code: -32097 
  Activating project at `~/.vscode/extensions/julialang.language-julia-1.54.2/scripts/environments/languageserver/v1.10`

What version of macos are you using?

If you have xcode installed, can you please try to run the following commands

echo 'int main(void) { return 0; }' | clang -x c -o main.o -
/Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld -flavor darwin -arch x86_64 -platform_version macos 13.3.1 13.3 -o main main.o

The OS is 10.14.6 (Mojave).

and the command returns this:

geo145:~ imac$ echo 'int main(void) { return 0; }' | clang -x c -o main.o -
geo145:~ imac$ /Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld -flavor darwin -arch x86_64 -o main main.o
dyld: Library not loaded: @rpath/libLLVM.dylib
  Referenced from: /Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld
  Reason: image not found
Abort trap: 6

Right, sorry, we need to set DYLD_FALLBACK_LIBRARY_PATH, could you please try

echo 'int main(void) { return 0; }' | clang -x c -o main.o -
DYLD_FALLBACK_LIBRARY_PATH="/Applications/Julia-1.10.app/Contents/Resources/julia/lib/julia:/Applications/Julia-1.10.app/Contents/Resources/julia/lib" /Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld -flavor darwin -arch x86_64 -platform_version macos 13.3.1 13.3 -o main main.o

It also produces “image not found”

geo145:~ imac$ echo 'int main(void) { return 0; }' | clang -x c -o main.o -
geo145:~ imac$ DYLD_FALLBACK_LIBRARY_PATH="/Applications/Julia-1.10.app/Contents/Resources/julia/lib/julia:/Applications/Julia-1.10.app/Contents/Resources/julia/lib" /Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld -flavor darwin -arch x86_64 -o main main.o
dyld: Library not loaded: @rpath/libLLVM.dylib
  Referenced from: /Applications/Julia-1.10.app/Contents/Resources/julia/libexec/julia/lld
  Reason: image not found
Abort trap: 6

Uhm, do you have SIP enabled? Can you try to temporary disable it following the instructions at Disabling and Enabling System Integrity Protection | Apple Developer Documentation and try again? Edit: but lld shouldn’t fall inside SIP cover, so I’m not sure it’s that one.

Problem solved after updating to OS Catalina (10.15).

1 Like