In case anyone else comes across this problem with the LSP, I found a workaround here LSP "Missing reference" woes - #16 by ffevotte.
I think the problem is associated with this unresolved issue Hover help doesn't show for own modules · Issue #2719 · julia-vscode/julia-vscode · GitHub.