REPL help mode discoverability

As a relative newcomer to Julia, I have been making increasing use of the REPL and other versions of tooling to understand and select a workflow for julia use. One problem I have is the ability to discover needed functions/methods/packages while working:

  1. Package documentation is largely available online with limited ability to connect from the julia REPL (at least on my win10 julia-1.5.3)
  2. Searching for a word with the “?word” help mode in the REPL doesn’t find “word” if the package/module defining it has not been loaded
  3. Even if “using package” has been done, “word” doesn’t seem to be found unless exported
  4. When method documentation is presented, there is no explicit module/package listed

Some ideas for possible improvements:

  1. Local searchable package documentation (see this thread)
  2. Ditto
  3. This seems to be possible given that we have loaded the package, it would entail a more aggressive search for docs to methods exported and then to non-exported. Perhaps ordering by exported followed by non-exported in the display.
  4. It would be a huge help if the one-line method/type/object description also included the module/package from which the documentation came. With all the multi-methods in Julia, knowing which module the method is documenting could greatly improve the utility of the REPL documentation.

Another Julia REPL wish:

Would it be possible to enable click-to-follow URL links in the terminal/console window of the REPL (including various modes)? That would be a great help with 1 and 2 above especially if 4 were implemented so one could directly connect to the relevant package documentation?

Apologies if this reads like a list of work for “other people” to do. I’m still coming up the learning curve for contribution to julia directly but from previous open source experience getting this type of feedback from new users can lead to a significantly improved julia experience. Once I get to the stage of being able to readily help with this type of thing, I’ll have trouble remembering what or that there was a problem at all.

I call this the problem of figuring out things that are “well known to those who know it well” which means it can be very difficult to even search because it is “obvious” to those who know so one can often not even find a discussion of the problem so that you can figure out what to call the problem to search for how to solve it. (Add tail recursion to this as needed…)

6 Likes

You might find the function apropos useful, it searches all installed docstrings from the REPL

4 Likes

In my use so far, apropos seemed not very helpful by default. In light of my post, I started from scratch:

help?> apropos
search: apropos hasproperty

apropos(string)

Search through all documentation for a string, ignoring case.

So now I try apropos(“run”) from the REPL prompt:

julia> apropos(“run”)
Core.@cmd
Base.shell_escape_winsomely
Base.IOBuffer
Base.redirect_stdout
Core.String

…many, many lines of output…

Pluto.run_single!
Pluto.TopologicalOrder
Pluto.is_just_text

which is not helpful to me and I don’t know which of the many options could be relevant to my interest. However, I have a couple of ideas for that :slight_smile:

  1. Maybe apropos(“word”) could be done by “??word” from the REPL prompt (or as “?word” from the help prompt.
  2. The above output could be extended with a second field containing the first line of context in the docstring where the search matched (and maybe with highlighted search term) if it is in that first line. This could be made more useful if there were a convention for a short one line description of each method/function/object.

Thanks for the reply.

2 Likes

This might help: JuliaHub
It enables search within all the registered packages for function names, documentation and more.

1 Like

Consider the contrast between

and

This is a fundamental problem with fuzzy search: it is difficult to find a balance between relevance and breadth. Good solutions require sophisticated algorithms and well-designed interfaces to present and navigate the results, which is tricky in a text terminal on a local machine.

I would seocnd @Iulian.Cioarca’s recommendation for JuliaHub, it was designed to solve precisely this problem.

1 Like

My understanding of the current approach for Julia package documentation is based on web sites and github pages.

These ideas were to improve the usability and discoverability of documentation in the Julia REPL. They were not code for “How do I find Julia documentation on the web?”

Here are the specific points from above (trying to be more explicit this time):

  • In 3) above, the idea is that docstrings would be searched for exported and internal objects of modules that have been loaded.
  • In 4) above, have the docs give the package/module information with the documentation for the object of interest. That would let you know what to search for (on juliahub for example). The package information could include the URL for the web documentation.
  • Add a link to the web documentation that can be clicked to follow from the REPL to the URL docs on the web.

Regarding “apropos” and “fuzzy search”: I understand the difficulties with finding and presenting information in response to a query. The thoughts for the REPL were

  • Add “??word” as an extension to quickly run “apropos(word-string)” from the REPL
  • Present a short description of the item with the name so there is additional context to the user to help decide which specific item from the output is most relevant for further search refinement.
  • Make a way to search the one-line descriptions rather than the brute force “grep” being done by apropos. This arguably would require more general changes in the expectations for julia docstrings and documentation in general.

The first idea for apropos was a shortcut to access it without addressing the limitations of the grep-based search output. The second idea could help fuzzy search content but is not a “simple” fix.

I believe that the 3 ideas for the “help” command could greatly improve the ability for users to access the Julia documentation: both docstrings and package documentation (which is largely on the web).

I would be careful about internal methods etc; searching them wholesale is most likely to just decrease the signal/noise ratio.

note that ?"text" already does this, but perhaps it is not well-known. Cf

This would be useful, but for “brief” and “extended” docs, as introduced in

6 Likes

note that ?“text” already does this, but perhaps it is not well-known. Cf

Thanks! I tried it without the quotes before. Is there a reason we have to use quotes there?

W/o the quotes, it takes a symbol (a function name, etc).

It would be nice if Julia packages were routinely installed along with their Documenter-generated HTML documentation, such that module documentation is always easily available locally via a file:// URL in an HTML browser, does not require github.com to be reachable, does not require package authors to set up github.com CI individually, and matches automatically the version of the module used. But there isn’t yet a standard mechanism for automatically including or building the HTML documentation in a package, or agreement on where it should be built and stored (e.g., committed on main branch, committed on separate HTML branch, built in CI, built locally, artifact, etc.). See also

https://github.com/JuliaLang/julia/issues/34278

2 Likes

Personally, I think we could do more on this front too; I don’t always like to rely on a browser or even an internet connection. Since vscode indexes your main environment, it would seem possible to write a search-my-environment function.

@devel-chm, one of the things you will soon discover about Julia, if you haven’t already, is that it’s possible to do interesting & useful things yourself. If you’re interested in working on it, perhaps start with
https://github.com/julia-vscode/SymbolServer.jl

4 Likes

Thanks for the feedback, @tim.holy.

It would be nice if it were possible to have the help mode search Base and Standard Library without using every included package. For example, I knew from discourse that the readdlm routine could be used. However, “?readdlm” gave nothing. Fortunately, the apropos(“readdlm”) output was manageable—this time.