[`foo`](@ref) to auto-link the Julia manual on discourse?

Meta Discussion
1

I find myself linking to the Julia manual a lot when answering questions on Discourse, and it would be nice to make that more convenient.

Is there any chance of supporting Documenter.jl style [`foo`](@ref) links to the Julia manual?

6 Likes
2

But which version of the docs? :wink:

(I often stumble across Git(Lab) issues which link to lines in the source code and people who posted them forgot/ignored that there are “permalinks” to make them valid forever. Otherwise those links can become obsolete or even misleading very quickly)

2 Likes
3

I don’t think this is possible. The tools in our toolbox here are pretty limited, and I think Documenter’s @ref needs something fairly advanced.

There are really only two tools we have:

  • We have the Auto linkify words theme component that simply adds a link underneath text that you wrote — like ModelingToolkit.jl — without needing to explicitly write out the [](...) markdown. But that doesn’t change the text that appears, it just adds a link to it. You can use regexes there.

  • We also have the builtin “Replace” functionality from “Watched Words”, which can actually replace text that’s written, but as our site is currently hosted it doesn’t even support regexes or matches. It’s just a straight-up plain-text find/replace.

But then there’s the bigger question about packages: How could I know how to link to CSV.read or Random.rand!? Perhaps there’s a missing Documenter.jl (and maybe DocumentationGenerator.jl) feature here: provide a URL endpoint like example.com/base/path/to/docs/stable/api/Module.function that will redirect to the place where that docstring lives.

4

How about a bot? Could we have an automated user that would reply with links to the documentation?

5

That sounds very noisy. We could have a @doc user with sufficient privileges to edit others’ posts… in fact the @system user already edits posts in some contexts (see this post’s very edit history).

But the general problem remains, how do you know what to do? How do you know Base.rand lives at stdlib/Random/#Base.rand while Base.zeros is base/arrays/#Base.zeros? Or that the CSV.read docstring is on stable/reading.html#CSV.read while CSV.write is at stable/writing.html#CSV.write…

1 Like
6

I might be misunderstanding the question. I thought this information is encoded in a JSON “database” somewhere such as https://docs.julialang.org/en/v1/search_index.js

{
  "location": "base/arrays/#Base.zeros",
  "page": "Arrays",
  "title": "Base.zeros",
  "text": "zeros([T=Float64,] dims::Tuple)\nzeros([T=Float64,] dims...)\n\nCreate an Array, with element type T, of all zeros with size specified by dims. See also fill, ones, zero.\n\nExamples\n\njulia> zeros(1)\n1-element Vector{Float64}:\n 0.0\n\njulia> zeros(Int8, 2, 3)\n2Ă—3 Matrix{Int8}:\n 0  0  0\n 0  0  0\n\n\n\n\n\n",
  "category": "function"
}

Or for CSV, https://docs.juliahub.com/General/CSV/stable/search_index.js:

{
  "location": "writing.html#CSV.write",
  "page": "Writing",
  "title": "CSV.write",
  "text": "CSV.write(file, table; kwargs...) => file\ntable |> CSV.write(file; kwargs...) => file\n\nWrite a Tables.jl interface input to a csv file, given as an IO argument or String/FilePaths.jl type representing the file name to write to. Alternatively, CSV.RowWriter creates a row iterator, producing a csv-formatted string for each row in an input table.\n\nSupported keyword arguments include:\n\nbufsize::Int=2^22: The length of the buffer to use when writing each csv-formatted row; default 4MB; if a row is larger than the bufsize an error is thrown\ndelim::Union{Char, String}=',': a character or string to print out as the file's delimiter\nquotechar::Char='\"': ascii character to use for quoting text fields that may contain delimiters or newlines\nopenquotechar::Char: instead of quotechar, use openquotechar and closequotechar to support different starting and ending quote characters\nescapechar::Char='\"': ascii character used to escape quote characters in a text field\nmissingstring::String=\"\": string to print for missing values\ndateformat=Dates.default_format(T): the date format string to use for printing out Date & DateTime columns\nappend=false: whether to append writing to an existing file/IO, if true, it will not write column names by default\ncompress=false: compress the written output using standard gzip compression (provided by the CodecZlib.jl package); note that a compression stream can always be provided as the first \"file\" argument to support other forms of compression; passing compress=true is just for convenience to avoid needing to manually setup a GzipCompressorStream\nwriteheader=!append: whether to write an initial row of delimited column names, not written by default if appending\nheader: pass a list of column names (Symbols or Strings) to use instead of the column names of the input table\nnewline='\\n': character or string to use to separate rows (lines in the csv file)\nquotestrings=false: whether to force all strings to be quoted or not\ndecimal='.': character to use as the decimal point when writing floating point numbers\ntransform=(col,val)->val: a function that is applied to every cell e.g. we can transform all nothing values to missing using (col, val) -> something(val, missing)\nbom=false: whether to write a UTF-8 BOM header (0xEF 0xBB 0xBF) or not\npartition::Bool=false: by passing true, the table argument is expected to implement Tables.partitions and the file argument can either be an indexable collection of IO, file Strings, or a single file String that will have an index appended to the name\n\nExamples\n\nusing CSV, Tables, DataFrames\n\n# write out a DataFrame to csv file\ndf = DataFrame(rand(10, 10), :auto)\nCSV.write(\"data.csv\", df)\n\n# write a matrix to an in-memory IOBuffer\nio = IOBuffer()\nmat = rand(10, 10)\nCSV.write(io, Tables.table(mat))\n\n\n\n\n\n",
  "category": "function"
}
1 Like
7

Ah, perfect, that is exactly the answer to my question there! It’s still something that’d require near-turing-completeness and a bot — it couldn’t be implemented with regexmatches, for example.

8

Here is some code to lookup the locations.

using HTTP, JSON3

function get_function_search_index(url::AbstractString="https://docs.julialang.org/en/v1/search_index.js")
    search_index = String(HTTP.get(url).body)
    search_index_json = JSON3.read(@view(search_index[29:end]))
    functions = filter(x->x.category == "function", search_index_json.docs)
    return functions
end

const julia_idx = get_function_search_index()

function find_function_location(title::AbstractString, idx::Vector{JSON3.Object} = julia_idx)
    only(filter(x->x.title==title, idx))
end

function get_function_url(
    title::AbstractString,
    idx::Vector{JSON3.Object} = julia_idx,
    prefix::AbstractString = "https://docs.julialang.org/en/v1/"
)
    return prefix * find_function_location(title, idx)
end

Here is a demonstration:

julia> find_function_location("Base.zeros", idx)
"base/arrays/#Base.zeros"

julia> find_function_location("Base.ones", idx)
"base/arrays/#Base.ones"

julia> find_function_location("Base.println", idx)
"base/io-network/#Base.println"

julia> get_function_url("Base.println")
"https://docs.julialang.org/en/v1/base/io-network/#Base.println"
1 Like
9

Indeed, Documenter should generate a downloadable index of docstrings: At-ref proposal with InterSphinx compatibility · Issue #2366 · JuliaDocs/Documenter.jl · GitHub

With the limitations of the Discourse site that you describe, I’m not sure how far that would get you, but it would be a first step.

10

There is now an inventory file for the Julia 1.10 documentation at Inventory File Repository · JuliaDocs/DocumenterInterLinks.jl Wiki · GitHub

It can be used programmatically (or explored in the REPL) via the DocInventories package – or just with a TOML parser.

2 Likes