Neovim + LanguageServer.jl

I keep getting this error :frowning:

⠸ [00m:14s] PackageCompiler: compiling incremental system imageSystemError: opening file "/home/davibarreira/.julia/environments/nvim-lspconfig/packagecompiler/precompile_statements.jl": No such file or directory
Stacktrace:
⡆ [00m:14s] PackageCompiler: compiling incremental system imageString, errno::Int32; extrainfo::Nothing)
    @ Base ./error.jl:174
  [2] #systemerror#68
    @ ./error.jl:173 [inlined]
  [3] systemerror
    @ ./error.jl:173 [inlined]
  [4] open(fname::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
    @ Base ./iostream.jl:293
  [5] open
    @ ./iostream.jl:282 [inlined]
  [6] #eachline#390
    @ ./io.jl:1014 [inlined]
  [7] eachline(filename::String)
    @ Base ./io.jl:1014
  [8] top-level scope
    @ /tmp/jl_GltfQx:73
  [9] eval(m::Module, e::Any)
    @ Core ./boot.jl:373
 [10] top-level scope
    @ /tmp/jl_GltfQx:61
in expression starting at /tmp/jl_GltfQx:61
⠋ [00m:15s] PackageCompiler: compiling incremental system image
ERROR: failed process: Process(`/opt/julia-1.7.0/bin/julia --color=auto --startup-file=no --cpu-target=native -O3 --sysimage=/opt/julia-1.7.0/lib/julia/sys.so --project=/home/davibarreira/.julia/environments/nvim-lspconfig --output-o=/tmp/jl_Lok4MU.o /tmp/jl_GltfQx`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:531 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:446
 [3] run
   @ ./process.jl:444 [inlined]
 [4] #14
   @ ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:157 [inlined]
 [5] spin(f::PackageCompiler.var"#14#15"{Cmd}, s::PackageCompiler.TerminalSpinners.Spinner{Base.TTY})
   @ PackageCompiler.TerminalSpinners ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:164
 [6] macro expansion
   @ ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:157 [inlined]
 [7] create_sysimg_object_file(object_file::String, packages::Vector{String}, packages_sysimg::Set{Base.PkgId}; project::String, base_sysimage::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, extra_precompiles::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:356
 [8] create_sysimage(packages::Symbol; sysimage_path::String, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::String, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:507
 [9] top-level scope
   @ none:1
make: *** [Makefile:6: languageserver.so] Error 1

@devmotion , the Lsp works, but not the sysimage script in the Makefile. Did you also follow the instructions for creating the image sys?

Yes, I did. I actually followed these instructions, seems unfortunately I copied the wrong link in my comment above.

My julia setup is brand new, since I just changed to 1.7. It had worked before, with my previous version :confused:

Ok, it worked now. It seems the script for the Makefile requires the vim setup before creating the sysimage. Thanks again @fredrikekre

1 Like

With conversation from @fredrikekre and @jvaverka, here is an updated version that works with Julia 1.8 and a Packer configuration. Please note that Fredrik is still the principal author and creator of this process - I just added some clarification on the process I took, made some steps more explicit, and made note of some issues I ran into:

  1. Install Mason.nvim or nvim-lspconfig and use them to install julials (it may also be called something like Julia Language Server Protocol).

  2. Modify init.vim or init.lua to use a custom Julia executable (if it exists):

require'lspconfig'.julials.setup{
    on_new_config = function(new_config, _)
        local julia = vim.fn.expand("~/.julia/environments/nvim-lspconfig/bin/julia")
        if require'lspconfig'.util.path.is_file(julia) then
	    vim.notify("Hello!")
            new_config.cmd[1] = julia
        end
    end
}

(OPTIONAL) If you use Packer to manage your vim setup, run PackerCompile at this stage.

NOTE: If you notice, there is a small line named vim.notify("Hello!"). This is to test that julials is engaged when accessing a Julia file - you can check that it is engaged by writing :messages in vim. You should see “Hello!” appear. This line can then safely be removed.

  1. Create the nvim-lspconfig Julia environment by running the following in your shell:
julia --project=~/.julia/environments/nvim-lspconfig -e 'using Pkg; Pkg.add("LanguageServer")'

And then navigate to the directory at ~.julia/environment/nvim-lspconfig.

  1. Copy the following makefile (courtesy of Fredrik Ekre) to the nvim-lspconfig directory with the name makefile:
# MIT License. Copyright (c) 2021 Fredrik Ekre
#
# This Makefile can be used to build a custom Julia system image for LanguageServer.jl to
# use with neovims built in LSP support. An up-to date version of this Makefile can be found
# at https://github.com/fredrikekre/.dotfiles/blob/master/.julia/environments/nvim-lspconfig/Makefile
#
# Usage instructions:
#
#   1. Update the neovim configuration to use a custom julia executable. If you use
#      nvim-lspconfig (recommended) you can modify the setup call to the following:
#
#          require("lspconfig").julials.setup({
#              on_new_config = function(new_config, _)
#                  local julia = vim.fn.expand("~/.julia/environments/nvim-lspconfig/bin/julia")
#                  if require("lspconfig").util.path.is_file(julia) then
#                      new_config.cmd[1] = julia
#                  end
#              end,
#              -- ...
#          })
#
#   2. Place this Makefile in ~/.julia/environments/nvim-lspconfig (create the directory if
#      it doesn't already exist).
#
#   3. Change directory to ~/.julia/environments/nvim-lspconfig and run `make`. This will
#      start up neovim in a custom project with a julia process that recods compiler
#      statements. Follow the instructions in the opened source file, and then exit neovim.
#
#   4. Upon exiting neovim PackageCompiler.jl will compile a custom system image which will
#      automatically be used whenever you work on Julia projects in neovim.
#
# Update instructions:
#
#  To update the system image (e.g. when upgrading Julia or upgrading LanguageServer.jl or
#  it's dependencies) run the following commands from the
#  ~/.julia/environments/nvim-lspconfig directory:
#
#      julia --project=. -e 'using Pkg; Pkg.update()'
#      make

JULIA=$(shell which julia)
JULIA_PROJECT=
SRCDIR:=$(shell dirname $(abspath $(firstword $(MAKEFILE_LIST))))
ifeq ($(shell uname -s),Linux)
	SYSIMAGE=languageserver.so
else
	SYSIMAGE=languageserver.dylib
endif

default: $(SYSIMAGE)

$(SYSIMAGE): Manifest.toml packagecompiler/Manifest.toml packagecompiler/precompile_statements.jl
	JULIA_LOAD_PATH=${PWD}:${PWD}/packagecompiler:@stdlib ${JULIA} -e 'using PackageCompiler; PackageCompiler.create_sysimage(:LanguageServer, sysimage_path="$(SYSIMAGE)", precompile_statements_file="packagecompiler/precompile_statements.jl")'

Manifest.toml: Project.toml
	JULIA_LOAD_PATH=${PWD}/Project.toml:@stdlib ${JULIA} -e 'using Pkg; Pkg.instantiate()'

Project.toml:
	JULIA_LOAD_PATH=${PWD}/Project.toml:@stdlib ${JULIA} -e 'using Pkg; Pkg.add("LanguageServer")'

packagecompiler/Manifest.toml: packagecompiler/Project.toml
	JULIA_LOAD_PATH=${PWD}/packagecompiler/Project.toml:@stdlib ${JULIA} -e 'using Pkg; Pkg.instantiate()'

packagecompiler/Project.toml:
	mkdir -p packagecompiler
	JULIA_LOAD_PATH=${PWD}/packagecompiler/Project.toml:@stdlib ${JULIA} -e 'using Pkg; Pkg.add("PackageCompiler")'

packagecompiler/precompile_statements.jl: Manifest.toml bin/julia
	TMPDIR=$(shell mktemp -d) && \
	cd $${TMPDIR} && \
	JULIA_LOAD_PATH=: ${JULIA} -e 'using Pkg; Pkg.generate("Example")' 2> /dev/null && \
	cd Example && \
	JULIA_LOAD_PATH=$${PWD}:@stdlib ${JULIA} -e 'using Pkg; Pkg.add(["JSON", "fzf_jll", "Random", "Zlib_jll"])' 2> /dev/null && \
	JULIA_LOAD_PATH=$${PWD}:@stdlib ${JULIA} -e 'using Pkg; Pkg.precompile()' 2> /dev/null && \
	echo "$$PACKAGE_CONTENT" > src/Example.jl && \
	JULIA_TRACE_COMPILE=1 nvim src/Example.jl && \ # NOTE: You may need to check that neovim is correctly on your path
	rm -rf $${TMPDIR}

bin/julia:
	mkdir -p bin
	echo "$$JULIA_SHIM" > $@
	chmod +x $@

clean:
	rm -rf $(SYSIMAGE) packagecompiler bin

.PHONY: clean default

export JULIA_SHIM
define JULIA_SHIM
#!/bin/bash
JULIA=${JULIA}
if [[ $${JULIA_TRACE_COMPILE} = "1" ]]; then
    exec $${JULIA} --trace-compile=${PWD}/packagecompiler/precompile_statements.jl "$$@"
elif [[ -f ${PWD}/$(SYSIMAGE) ]]; then
    exec $${JULIA} --sysimage=${PWD}/$(SYSIMAGE) "$$@"
else
    exec $${JULIA} "$$@"
fi
endef

export PACKAGE_CONTENT
define PACKAGE_CONTENT
# This file is opened in neovim with a LanguageServer.jl process that records Julia
# compilation statements for creating a custom sysimage.
#
# This file has a bunch of linter errors which will exercise the linter and record
# statements for that. When the diagnostic messages corresponding to those errors show up in
# the buffer the language server should be ready to accept other commands (note: this may
# take a while -- be patient). Here are some suggestions for various LSP functionality that
# can be exercised (your regular keybindings should work):
#
#  - :lua vim.lsp.buf.hover()
#  - :lua vim.lsp.buf.definition()
#  - :lua vim.lsp.buf.references()
#  - :lua vim.lsp.buf.rename()
#  - :lua vim.lsp.buf.formatting()
#  - :lua vim.lsp.buf.formatting_sync()
#  - :lua vim.lsp.buf.code_action()
#  - Tab completion (if you have set this up using LSP)
#  - ...
#
# When you are finished, simply exit neovim and PackageCompiler.jl will use all the recorded
# statements to create a custom sysimage. This sysimage will be used for the language server
# process in the future, and should result in almost instant response.

module Example

import JSON
import fzf_jll
using Random
using Zlib_jll

function hello(who, notused)
    println("hello", who)
    shuffle([1, 2, 3])
   shoffle([1, 2, 3])
    fzzf = fzf_jll.fzzf()
    fzf = fzf_jll.fzf(1)
    JSON.print(stdout, Dict("hello" => [1, 2, 3]), 2, 123)
    JSON.print(stdout, Dict("hello" => [1, 2, 3]))
    hi(who)
    return Zlib_jll.libz
end

function world(s)
    if s == nothing
      hello(s)
  else
      hello(s)
  end
    x = [1, 2, 3]
    for i in 1:length(x)
        println(x[i])
    end
end

end # module
endef
  1. Run make. This will set up a dummy project and launch nvim with julia recording everything that is compiled. Wait until the LanguageServer responds (there are a bunch of things in this dummy project that will result in warnings) and then run some LanguageServer commands, for example ::lua vim.lsp.buf.hover() to fetch documentation).

  2. Quit vim.

  3. PackageCompiler will now build a custom languageserver.so sysimage.

  4. Enjoy the Julia LSP!

7 Likes

Rather than make a new thread, I thought I’d post here, since I think several of the posts above touch on what’s likely the answer.

I’m trying to get Neovim working with mason.nvim (well, properly speaking lsp-zero.nvim, which uses mason.nvim as the base for installing the LSP server).

Things seem to have been installed correctly, but every time I load a julia file (which should start the LSP server, right?), I get the error message “Spawning language server with cmd: ‘julia-lsp’ failed. The language server is either not installed, missing from PATH, or not executable.”

I have installed a separate environment for the lsp server in ~\.julia\environments\nvim-lspconfig. and Mason.nvim (via :MasonLog) shows the LSP was installed successfully. I do notice that when I call LspInfo, it says my “root directory” is “not found,” if that means anything.

When I look at the log, here’s what I see:

[START][2022-09-27 08:39:46] LSP logging initiated
[INFO][2022-09-27 08:39:46] …/vim/lsp/rpc.lua:261 “Starting RPC client” { args = { “”, “C:\Users\opera\.julia\environments\v1.8” }, cmd = “julia-lsp”, extra = { cwd = “C:/Users/opera” }}

At this point, I’ve been trying to get this working for about two days now, and I’m about to tear my hair out.

I’m sure I must be missing something obvious, although googling hasn’t helped. Any advice?

1 Like

This is a mistake that I did at one point - do you have this in your init.lua somewhere for LSP-config @opera_malenky ?

What I found after installing the LSP was I needed the above snippet and then to run PackerCompile to compile my environment and then to finally create reopen neovim to a Julia file. Any luck?

Also, you are welcome to look at my config which is here - particularly the folder lua/config and the lsp lua files may be helpful: GitHub - TheCedarPrince/Suffice: My Neovim configuration

Thanks! I wasn’t even trying for the precompiled version right now. :smiley: Just trying to get the vanilla version running.

I caught a typo in the script I’d been trying, and it seems to be working now.

In case anyone runs across this in the future, I’ll leave the relevant snippet from my init.lua file that uses lsp-zero.nvim (which bootstraps configurations for both mason.nvim and nvim-cmp):

-- LSP and autocompletion via lsp-zero
local lsp = require('lsp-zero')
lsp.preset('recommended')
lsp.ensure_installed({'julials'})
lsp.configure('julials', {
    julia_env_path = {"C:\\Users\\opera\\.julia\\environments\\nvim-lspconfig"}
    }
)

lsp.setup()

The problem I was having seems to be that the environment path was not being set properly.

mason.nvim uses nvim-lspconfig, which AFAIK is supposed to set the julia_env_path variable to the one I have above, and use that to run the julia command.

Apparently, that default was not being set correctly out of the box, somehow. Perhaps mason.nvim was not using the nvim-lspconfig defaults, not sure. However, if I manually set the environment path, as above, things work fine for both single files and projects (neither of which had worked before).

Anyway, thanks for taking the time to respond!

1 Like

Oh weird! However, glad to hear you have gotten things fixed up on your side – I half-wondered if there could’ve been a typo as it sounded like you did everything right. Glad things got resolved and happy vimming! Go forth and conquer my friend!

1 Like

Thanks for this!

I will note that after step 5 (Quit vim), I saw this error

/bin/sh: line 8:  #: command not found
make: *** [makefile:69: packagecompiler/precompile_statements.jl] Error 127

I couldn’t figure out what caused the error, so I just ran make again, and it seemed to work the second time.

1 Like

" #: command not found" error happens because in the makefile you have a comment with a space before it.

I don’t know if it is an intentional way of working of makefile-s but for me that was the issue.

1 Like

Oh where exactly? I can update instructions. Thanks!

1 Like

The project that opened automatically after running make worked great, I saw all the diagnostics, warnings and errors. But when I open some other julia file, the LSP (julials) starts, initializes, but not everything seems to work.

The auto-completion and suggestions work fine, also the references and go-to-definition work just fine, formatting is working as well. The pop-up documentation works fine as well. But I can’t see the inline-warnings and diagnostics or any diagnostics at all, just syntax highlighting.

One suspicious thing that comes to mind is when I type “:LspInfo” there is “Running in single-file mode”. No errors, nothing red, everything seem to be working fine. So I guess I have some configuration problem, not “something is broken” problem.

Any idea what I might be doing wrong? Thanks

1 Like

Holy cow, that helped! Thank you! Now I can see diagnostics as well as other LSP stuff.

Thanks for all of your help. It saved me tearing out any more of my (very limited) hair.

1 Like

Now that packages are precompiled and persistent among Julia sessions (v1.10), could this workflow be improved to get rid of the sysimage?

1 Like

@fredrikekre thoughts on this?

In my experience, a sysimage still helps reduce latency for the languageserver startup in Neovim.