How do I enable ANSI colored text output within Documenter.jl?

Hi, I’m currently developing a package in which coloring plays an important role.

The HTML output of Documenter.jl writes raw ANSI escape codes for text output such as @example as shown in the upper block in the screenshot. (Of course, the :color property of IOContext is not set, so ANSI escape codes are not usually displayed.)

I would like to get the ANSI colored output as shown in the lower block. Also, I don’t want to implement terminal emulation on the package (i.e. src, not docs) side, because the package will support both static “text/plain” and interactive “text/html” printing.

What are the best practices for ANSI color printing?

3 Likes

As you already observed, Documenter does not have any support for the ANSI colors at the moment. However, I do think it would be neat if it did (i.e. that if something can be printed in color, it will be).

It shouldn’t be too hard to get it working natively with Documenter, but it would need a little bit of work. A few breadcrumbs:

  1. The return values of at-example blocks get showed in the Expanders module, which uses these functions here. Handling of stdout/stderr is near there as well.
  2. We’d need to parse the ANSI sequences and figure out what style each character has. You already have a working implementation for this (AnsiToHTML)?
  3. HTMLWriter converts the text/plain into HTML by using our DOM representation. Here we’d just need to loop over the string and wrap all the characters in <span>s I think.

What needs some thought is whether/how we should handle both colored and uncolored output simultaneously.

4 Likes

Thanks for the helpful references.

I have ~200 lines of “rough” code supporting SGR code 0-9, 22, (Edit: 23-25, 27-29), 30-39, 40-49, 90-97, 100-107.

I guess such code already exists somewhere, but I couldn’t find it in JuliaHub package search.
(Although I know there are some JavaScript libraries, I prefer a static conversion on this one.)

If we don’t have a standard tool, I’ll release the code.

BTW, while the state management shares some similarities with Crayons.jl, my code does not rely on Crayons.jl at the moment, because there are some differences in the concept of stack management.

1 Like

I haven’t written the test code yet. :sweat_smile:
Edit: It has been transferred from my personal repository to JuliaDocs.
https://github.com/JuliaDocs/ANSIColoredPrinters.jl

Documentation with examples (dev):
https://juliadocs.github.io/ANSIColoredPrinters.jl/dev/

Package design issue:
https://github.com/JuliaDocs/ANSIColoredPrinters.jl/issues/4

3 Likes

Now, I’ve tried the following changes.
(WIP: Comparing JuliaDocs:master...kimikage:ansicolor · JuliaDocs/Documenter.jl · GitHub)

  1. add a field / argument ansicolor to User / makedocs().
  2. add a keyword argument ansicolor in the @example block to override the user.ansicolor.
  3. add a context keyword argument to Utilities.display_dict().
function limitstringmime(m::MIME"text/plain", x; context=nothing)
    io = IOBuffer()
    ioc = IOContext(context === nothing ? io : IOContext(io, context), :limit => true)
    show(ioc, m, x)
    return String(take!(io))
end
function display_dict(x; context=nothing)
    out = Dict{MIME,Any}()
    x === nothing && return out
    # Always generate text/plain
    out[MIME"text/plain"()] = limitstringmime(MIME"text/plain"(), x, context=context)
    for m in [MIME"text/html"(), MIME"image/svg+xml"(), MIME"image/png"(),
              MIME"image/webp"(), MIME"image/gif"(), MIME"image/jpeg"(),
              MIME"text/latex"(), MIME"text/markdown"()]
        showable(m, x) && (out[m] = stringmime(m, x, context=context))
    end
    return out
end
  1. in Selectors.runner(::Type{ExampleBlocks}, x, page, doc), call:
output = Base.invokelatest(Utilities.display_dict, result, context=:color => ansicolor)
  1. in HTMLWriter.mdconvert(), use AnsiColoredPrinters.HTMLPrinter:
...
    elseif haskey(d, MIME"text/plain"())
        input = IOBuffer(d[MIME"text/plain"()])
        printer = AnsiColoredPrinters.HTMLPrinter(input, root_class="documenter-example-output")
        out = Documents.RawHTML(repr(MIME"text/html"(), printer))
...

The changes successfully colorize lazy-printing objects.

However, the stdout output does not have ANSI escape codes.

What should I do after that? In the meantime, should I submit a WIP PR?
cc: @mortenpi


BTW, which setting of which SASS/SCSS compiler should I use?

2 Likes

Scrolling through the diff quickly, this looks great! Please do open a PR and we can discuss the technical details there.

Regarding Sass, DocumenterTools can re-compile the themes for you (it will re-generate the .css files under assets/html/themes, using Sass.jl):

using DocumenterTools: Themes
Themes.compile_native_themes()
2 Likes

I opened a draft PR.
xref: https://github.com/JuliaDocs/Documenter.jl/pull/1441

(Due to troubles with the use of unregistered packages and automatic whitespace removal from CSS files by the IDE :scream:, I’ve only checked that it works locally yet. :sweat_smile:)

Also, thank you for telling me about compile_native_themes()!

1 Like

Update on this.

JuliaDocs/Documenter.jl#1441 is now merged, and the next Documenter version (0.27.4) will have color output! Here is a screenshot of the generated HTML:

Thanks @kimikage !

5 Likes

Sorry to bump an old thread, but are there any extra steps needed to get this to work? My docs are still showing up uncolored. Here is a UnicodePlot:

The markdown code is

```@example
using UnicodePlots, Crayons
scatterplot(rand(50), rand(50), color=crayon"magenta")
```

and my make.jl file reads

using Documenter
using PackageName

makedocs(
    sitename="PackageName",
    modules=[PackageName],
    pages=Any[
        # ...
    ],
)

Currently, the color capture is disabled by default (to maintain backwards compatibility; this should change in 0.28, which is why the docs already imply that it’s enabled).

You need to explicitly pass format = Documenter.HTML(ansicolor = true) to makedocs to enable it (or set @example; ansicolor = true for each block you want colored).

1 Like

I successfully got this to work if I build the documentation locally, but color is still disabled when the docs are build via github actions. Is there anything else to do to get this to work online?

If it works locally, it should work on GHA without any additional configuration. Without seeing the repo and CI logs, it’s hard to say anything more. I would double check that (1) the new version actually deployed, and (2) that the Documenter setup in the repository is actually fully up to date.

Okay, the last documentation build looks fine:
https://github.com/StructuralEquationModels/StructuralEquationModels.jl/actions/runs/2257064202
The make.jl file is here:

ansicolor = true is enabelled globally and also in the @example blocks.
For example, the outputs on this page should be colored
https://structuralequationmodels.github.io/StructuralEquationModels.jl/dev/tutorials/first_model/
that is specified here

For example, the last output on this page locally looks like this:

Interesting. As a test, could you add a --color=yes to the make.jl call on CI (i.e. change this to julia --color=yes --project=docs/ docs/make.jl)?

Edit: I tested this locally now. It looks like the coloring of the StructuralEquationModels output is sensitive to the --color option, so setting --color=yes on CI will likely fix this.

Cool, it works now! Thanks!