Language server - "Missing references" in files included in an examples folder

General Usage
1

Hi, I am developing a package to which I have added an example folder including some additional dependencies in a separate Project.toml file. The overall structure looks more or less like

├── examples
│   ├── first_example.jl
│   ├── Manifest.toml
│   └── Project.toml
├── Manifest.toml
├── Project.toml
├── README.md
└── src
    └── MyPackage.jl

For editing I use helix with the following language server configuration

[language-server.julia]
command = "julia"
args = [
 "--startup-file=no",
 "--history-file=no",
 "-e",
 """
 using Pkg;
 using LanguageServer;
 runserver()
 """,
 ]

While in MyPackage.jl the LS works as I would expect, in first_example.jl I keep getting Missing reference: ... warnings. If I activate the environment in my examples folder I can run first_example.jl without any issues so there must be something wrong with my LS configuration.

Interestingly enough, the LSP starts correctly in the examples folder and I do get [ Info: Package MyPackage (<UUID>) is cached along with all other dependencies at the start and [ Info: Loaded all packages into cache in 0.84s at the very end, so I have no idea what might be the problem.

2

This kind of workflow should be supported by LS, so there must indeed be something wrong with the way you set things up.

You didn’t provide much details about the way you created the environment in examples. Did you use something like Pkg.develop(path="..")?

Also, I’m not familiar with helix at all, but do you know in which directory it starts the language server? This is important because the language server current working directory will be used to determine the environment in which the language server operates.

It may not have anything to do with your issue, but seeing how short you LS invocation is, I can’t help wondering how it works. In which environment is LanguageServer installed?

3

Yes. I created an empty Project.toml file, started julia in the examples folder, activated the environment using ] activate ., added all other dependencies except MyPackage using ] add and then ran ] develop ..

It does seem to start in the current directory as there is a difference in the packages being loaded depending on if I start helix in MyPackage or MyPackage/examples.

LanguageServer should be installed in the main environment of the “user”. It is certainly not installed in MyPackage nor MyPackage/examples and to install it globally I expect sudo privileges would be required.

If there is a different invocation you’d recommend I can try it.

4

OK, this is what I do too; it should work.

If by “current directory” you mean the directory in which helix is started, this might be (part of) the problem. AFAIK LanguageServer.runserver() will try to determine the current project by

  1. looking in ARGS[1] (but here you don’t provide any command-line argument)
  2. working its way up from pwd() until it finds a Project.toml.

So you’d need to either ensure that LS is started in the directory of the file you’re editing. Or you could perhaps give an additional command-line arg to tell LS where to find the relevant Project.toml.

This might require some support from helix. For example, for emacs in eglot-jl all this logic is done in the emacs side of things: we try to determine which project the file currently being edited belongs to, and provide this to the LanguageServer.

This should be fine. But other ways are possible. For example, again in eglot-jl we have LanguageServer and SymbolServer installed in a small, dedicated environment. This makes it less likely that globally installed packages interfere with LS.

5

I am almost certain the LS starts in the correct directory and sees the Project.toml as I do get different (and correct) packages listed as cached — [ Info: Package <PackageName> (<UUID>) is cached — by the LS, depending on if I start helix in MyPackage or MyPackage/examples.

OK, this makes sense. I tried doing the same but I still get the Missing reference: warnings.

Maybe, just to make sure I didn’t mess-up somewhere in my project, this is the package where I’m trying to get it to work. So if you could open examples/cartpole_swing_up/optimization.jl and check that you don’t get any Missing reference warnings, the problem would really be somewhere in the config/LS/editor.

6

Yes, I can confirm that everything works on my side (Emacs with eglot-jl handling the LS stuff). What I did:

shell$ git clone https://github.com/JurajLieskovsky/IterativeLQR.jl/

shell$ cd IterativeLQR.jl/examples

shell$ julia --project
(examples) pkg> dev ..
   Resolving package versions...
   Installed FFMPEG ─ v0.4.2
   Installed GR_jll ─ v0.73.8+0
   Installed GR ───── v0.73.8
[...]

shell$ emacs cartpole_swing_up/optimization.jl

Of course, at first I did get Missing reference warnings all over the place, but once LS had some time to index everything, the warnings disappeared. Here is a screenshot of LS listing all references for IterativeLQR.nominal_trajectory

Screenshot_2024-10-24_23-45-54
Screenshot_2024-10-24_23-45-54826×750 63.2 KB

FYI, here is LanguageServer’s startup summary, showing it took around 40 seconds on my machine for it to get fully operational:

[stderr]  ============== Startup timings ==============
[stderr]         0.0 - LS startup started (0.0s since last event)
[stderr]   0.0080829 - connection established (0.0080829s since last event)
[stderr]    0.051268 - (async) listening to client events (0.043185s since last event)
[stderr]    0.083091 - (async) listening to symbol server events (0.031823s since last event)
[stderr]    0.083118 - starting combined listener (2.718e-5s since last event)
[stderr]     0.12621 - LSP/initialize (0.043089s since last event)
[stderr]      1.2725 - LSP/initialized (1.1463s since last event)
[stderr]      3.3708 - LSP/textDocument/didOpen (2.0983s since last event)
[stderr]      3.5914 - LSP/workspace/didChangeConfiguration (0.22061s since last event)
[stderr]      3.9803 - LSP/textDocument/signatureHelp (0.38892s since last event)
[stderr]      4.3679 - LSP/textDocument/hover (0.38756s since last event)
[stderr]      4.9816 - LSP/textDocument/documentHighlight (0.61376s since last event)
[stderr]      38.876 - symbols received (33.895s since last event)
[stderr]       38.88 - extended methods computed (0.0040059s since last event)
[stderr]       38.88 - project deps computed (1.4067e-5s since last event)
[stderr]       38.88 - env map computed (5.4121e-5s since last event)
[stderr]      39.063 - initial lint done (0.18256s since last event)
[stderr]  =============================================
7

There seem to be some extra steps performed in your LS startup, specifically

The odd thing is that I the difference is still there even after I used eglot-jl for my LS configuration

[language-server.julia]
command = "julia"
args = [
 "--project=/home/juraj/eglot-jl",
 "/home/juraj/eglot-jl/eglot-jl.jl"
]

I would assume, the only difference in the startup is that I don’t have a depot_path set

┌ Info: Running language server\n
│   env = \"/home/juraj/eglot-jl/Project.toml\"\n
│   src_path = \"/home/juraj/OneDrive/u12110/IterativeLQR/examples\"\n
│   project_path = \"/home/juraj/OneDrive/u12110/IterativeLQR/examples\"\n
└   depot_path = \"\"\n
8

Good idea to use eglot-jl’s julia script to launch the server inexactly the same conditions!

I wouldn’t worry about the depot path. It’s rarely used, and in any case I don’t set one either.

I would now assume that remaining differences come from the LSP client, i.e. Emacs’ (or VS Code’s) LSP implementation vs Helix’ implementation.

In case it would help investigating this, here is a pretty representation of the first request sent by eglot to initialize the language server:

{
  "method": "initialize",
  "id": 1,
  "params": {
    "capabilities": {
      "workspace": {
        "didChangeWatchedFiles": {
          "dynamicRegistration": true
        },
        "workspaceEdit": {
          "documentChanges": false
        },
        "applyEdit": true,
        "executeCommand": {
          "dynamicRegistration": false
        },
        "symbol": {
          "dynamicRegistration": false
        },
        "configuration": true
      },
      "experimental": {},
      "textDocument": {
        "publishDiagnostics": {
          "relatedInformation": false,
          "tagSupport": {
            "valueSet": [
              1,
              2
            ]
          },
          "codeDescriptionSupport": false
        },
        "signatureHelp": {
          "signatureInformation": {
            "parameterInformation": {
              "labelOffsetSupport": true
            },
            "activeParameterSupport": true
          },
          "dynamicRegistration": false
        },
        "rename": {
          "dynamicRegistration": false
        },
        "hover": {
          "contentFormat": [
            "markdown",
            "plaintext"
          ],
          "dynamicRegistration": false
        },
        "declaration": {
          "linkSupport": true,
          "dynamicRegistration": false
        },
        "references": {
          "dynamicRegistration": false
        },
        "documentHighlight": {
          "dynamicRegistration": false
        },
        "implementation": {
          "linkSupport": true,
          "dynamicRegistration": false
        },
        "synchronization": {
          "didSave": true,
          "willSaveWaitUntil": true,
          "willSave": true,
          "dynamicRegistration": false
        },
        "definition": {
          "linkSupport": true,
          "dynamicRegistration": false
        },
        "documentSymbol": {
          "symbolKind": {
            "valueSet": [
              1,
              2,
              3,
              4,
              5,
              6,
              7,
              8,
              9,
              10,
              11,
              12,
              13,
              14,
              15,
              16,
              17,
              18,
              19,
              20,
              21,
              22,
              23,
              24,
              25,
              26
            ]
          },
          "hierarchicalDocumentSymbolSupport": true,
          "dynamicRegistration": false
        },
        "codeAction": {
          "codeActionLiteralSupport": {
            "codeActionKind": {
              "valueSet": [
                "quickfix",
                "refactor",
                "refactor.extract",
                "refactor.inline",
                "refactor.rewrite",
                "source",
                "source.organizeImports"
              ]
            }
          },
          "isPreferredSupport": true,
          "dynamicRegistration": false
        },
        "completion": {
          "completionItem": {
            "deprecatedSupport": true,
            "snippetSupport": true,
            "tagSupport": {
              "valueSet": [
                1
              ]
            }
          },
          "contextSupport": true,
          "dynamicRegistration": false
        },
        "formatting": {
          "dynamicRegistration": false
        },
        "typeDefinition": {
          "linkSupport": true,
          "dynamicRegistration": false
        },
        "rangeFormatting": {
          "dynamicRegistration": false
        }
      }
    },
    "rootUri": "file:///path/to/project",
    "processId": 1475034,
    "initializationOptions": {},
    "rootPath": "/path/to/project/"
  },
  "jsonrpc": "2.0"
}
1 Like
9

The communication does seem somewhat different. This is the first request

{
   "jsonrpc":"2.0",
   "method":"initialize",
   "params":{
      "capabilities":{
         "general":{
            "positionEncodings":[
               "utf-8",
               "utf-32",
               "utf-16"
            ]
         },
         "textDocument":{
            "codeAction":{
               "codeActionLiteralSupport":{
                  "codeActionKind":{
                     "valueSet":[
                        "",
                        "quickfix",
                        "refactor",
                        "refactor.extract",
                        "refactor.inline",
                        "refactor.rewrite",
                        "source",
                        "source.organizeImports"
                     ]
                  }
               },
               "dataSupport":true,
               "disabledSupport":true,
               "isPreferredSupport":true,
               "resolveSupport":{
                  "properties":[
                     "edit",
                     "command"
                  ]
               }
            },
            "completion":{
               "completionItem":{
                  "deprecatedSupport":true,
                  "insertReplaceSupport":true,
                  "resolveSupport":{
                     "properties":[
                        "documentation",
                        "detail",
                        "additionalTextEdits"
                     ]
                  },
                  "snippetSupport":true,
                  "tagSupport":{
                     "valueSet":[
                        1
                     ]
                  }
               },
               "completionItemKind":{
                  
               }
            },
            "formatting":{
               "dynamicRegistration":false
            },
            "hover":{
               "contentFormat":[
                  "markdown"
               ]
            },
            "inlayHint":{
               "dynamicRegistration":false
            },
            "publishDiagnostics":{
               "tagSupport":{
                  "valueSet":[
                     1,
                     2
                  ]
               },
               "versionSupport":true
            },
            "rename":{
               "dynamicRegistration":false,
               "honorsChangeAnnotations":false,
               "prepareSupport":true
            },
            "signatureHelp":{
               "signatureInformation":{
                  "activeParameterSupport":true,
                  "documentationFormat":[
                     "markdown"
                  ],
                  "parameterInformation":{
                     "labelOffsetSupport":true
                  }
               }
            }
         },
         "window":{
            "workDoneProgress":true
         },
         "workspace":{
            "applyEdit":true,
            "configuration":true,
            "didChangeConfiguration":{
               "dynamicRegistration":false
            },
            "didChangeWatchedFiles":{
               "dynamicRegistration":true,
               "relativePatternSupport":false
            },
            "executeCommand":{
               "dynamicRegistration":false
            },
            "fileOperations":{
               "didRename":true,
               "willRename":true
            },
            "inlayHint":{
               "refreshSupport":false
            },
            "symbol":{
               "dynamicRegistration":false
            },
            "workspaceEdit":{
               "documentChanges":true,
               "failureHandling":"abort",
               "normalizesLineEndings":false,
               "resourceOperations":[
                  "create",
                  "rename",
                  "delete"
               ]
            },
            "workspaceFolders":true
         }
      },
      "clientInfo":{
         "name":"helix",
         "version":"24.7"
      },
      "processId":19453,
      "rootPath":"/home/juraj/OneDrive/u12110/IterativeLQR",
      "rootUri":"file:///home/juraj/OneDrive/u12110/IterativeLQR",
      "workspaceFolders":[
         {
            "name":"IterativeLQR",
            "uri":"file:///home/juraj/OneDrive/u12110/IterativeLQR"
         }
      ]
   },
   "id":0
}

and this the first response

{
   "id":0,
   "jsonrpc":"2.0",
   "result":{
      "capabilities":{
         "textDocumentSync":{
            "openClose":true,
            "change":2,
            "willSave":false,
            "willSaveWaitUntil":false,
            "save":{
               "includeText":true
            }
         },
         "completionProvider":{
            "resolveProvider":false,
            "triggerCharacters":[
               ".",
               "@",
               "\"",
               "^"
            ]
         },
         "hoverProvider":true,
         "signatureHelpProvider":{
            "triggerCharacters":[
               "(",
               ","
            ]
         },
         "declarationProvider":false,
         "definitionProvider":true,
         "typeDefinitionProvider":false,
         "implementationProvider":false,
         "referencesProvider":true,
         "documentHighlightProvider":true,
         "documentSymbolProvider":true,
         "codeActionProvider":true,
         "documentLinkProvider":{
            "resolveProvider":false
         },
         "colorProvider":false,
         "documentFormattingProvider":true,
         "documentRangeFormattingProvider":true,
         "renameProvider":{
            "prepareProvider":true
         },
         "foldingRangeProvider":false,
         "executeCommandProvider":{
            "commands":[
               "UpdateDocstringSignature",
               "CompareNothingWithTripleEqual",
               "AddDocstringTemplate",
               "FixMissingRef",
               "ReexportModule",
               "ReplaceUnusedAssignmentName",
               "ExpandFunction",
               "RewriteAsRawString",
               "ExplicitPackageVarImport",
               "DeleteUnusedFunctionArgumentName",
               "OrganizeImports",
               "AddLicenseIdentifier",
               "RewriteAsRegularString"
            ]
         },
         "selectionRangeProvider":true,
         "inlayHintProvider":true,
         "workspaceSymbolProvider":true,
         "workspace":{
            "workspaceFolders":{
               "supported":true,
               "changeNotifications":true
            }
         }
      }
   }
}
10

Edit: The issue does indeed seem to be in the workspace folder

I suppose the correct Project.toml file is found but helix starts the LS in the base folder of the package?

Remembering that helix likes to base its workspace based on the location the containing git repository (workspace directory seem to differ from the working directory which might explain why the correct Project.toml file was loaded) I created a copy of my package where I deleted the .git folder and now the LS works.

So I’ll have to figure out a way which does not require me to delete the .git folder but I think that’s specific to helix. @ffevotte thank you for all your help.

1 Like
11

The solution seems to be adding a .helix/config.toml file to the root of my package

# .helix/config.toml
[editor]
workspace-lsp-roots = ["examples"]

and then starting helix in the examples folder. This results in the LS starting with the examples folder as its “LSP root”.

Considering this I was wondering if I would have to use a similar solution for a potential test folder but to my surprise the LS works in this case even if the “LSP root” in helix remains the base directory of the package.

12

Test dependencies benefit from a special treatment, at least in Pkg. Maybe LS also handles them specifically?

However, you’re probably going to have to use the same kind of solution for your docs folder if you need LS to work in it.