A request for feedback and testing for a large piece of work I’ve undertaken
Background
The StyledStrings was first conceptualised in June 2021, PR’d in May 2023, merged in Oct 2023, and shipped in Julia v1.11.
Since then, the initial design has undergone several rounds of refinement. Lilith has been fantastic helping hash out some of the particular semantics, Sukera helped me use Supposition.jl’s PBT to make the styled""" markup syntax parsing robust, Kristoffer has identified and implemented a handful of performance wins, and most recently, Cody has been a big help in working out how to work around the Base/stdlib split and thorny invalidations caused by it.
While all of these improvements have been valuable additions, time has also revealed which of the decisions made in the original design/implementation have caused the biggest headaches and what compromises have struck a poor balance in hindsight.
I’ve recently pushed some work that’s been discussed on-and-off with various people to a viable state, to address all of the headaches and shortcomings that have emerged as major annoyances/impediments that I’m (a) aware of, and (b) feel confident to handle. This has taken a while in large part because I’ve been waiting for a few key design decisions to settle and feel “right” in my mind.
I’m now very interested in getting feedback on the proposed changes, and encouraging anybody interested to take the WIP branch for a spin and see what you think in practice/if you can find anything I’ve overlooked.
The headaches and shortcomings
- Piracy with the dispatch on printing/display of
AnnotatedString(defined inBase, implemented inStyledStrings) - The
AnnotatedStringbehaviour - It not being possible for any other package to define its own annotation type with custom display/printing
- A lack of namespacing
- The
Anyvalue type in annotations (AnnotatedString/AnnotatedChar/AnnotatedIO) - No validation of named faces (
styled"{bluue:text}"is silently unstyled) - Long prefixes when using custom faces within a package (e.g.
REPL_History_search_unselected) - Consistent face name prefixes relying on package authors always following the recommendations (in other words: inconsistent face naming)
- The use of
Faces directly mixed with the use ofSymbols to refer to named faces (and all theUnion{Symbol, Face}code that results) - The inability to remap faces, except when displaying (
withfaces) - The inability to unset a property of a face in user configuration (only set it to a different value)
In the end, about half of these items are collectively addressed in a suite of tightly connected changes that I’m collectively terming “Face primacy”. More on that later.
Now, onto the solutions/proposed changes.
Oh, by the way, everything here is fully backwards-compatible, up to Annotated{String,Char,IOBuffer} gaining an extra type parameter.
Parameterising the value of annotations
To remove the value::Any from the various annotated types, we change it to value::V where V. By itself, this wouldn’t be terribly helpful since we’d then have StyledStrings using a combination of face names (Symbols) and Face values: value::Symbol, value::Face, and value::Union{Symbol, Face}. As well as this seeming likely to make inference more complex, the use of Symbol prevents the StyledStrings display methods from being based solely on StyledStrings-defined types, which is what we need to avoid piracy.
Hybrid-interned Faces, instead of Symbols
With value-parameterised annotations, it would be simplest and nicest to just have value::Face, but eliminating Symbol-named Faces both makes the Public API much more inconvenient, and makes some detailes of the implementation very tricky (colours are implemented as named Faces, without names there is no “red” foreground/background).
The neat solution to this I’ve landed on is naming Faces with Faces. Specifically, by:
- Renaming the immutable
Facestruct - Creating a mutable wrapper type
- Interning
Faces by name (into anIdDict) - Creating a convenience for accessing/interning faces:
face"<name>"
This lets us drop the use of :red to refer to a particular Face (Symbol => Face), and instead use face"red" (Face => Face). Customisations etc. are referenced against that particular Face, instead of the name.
This is integrated into the styled""" macro, so that styled"{red:text}" uses the new interning system.
Note: there’s more going on here than just interning (hence the “hybrid”), as you’ll see in the next section.
Old Sample
importantface = :emphasis
astr = styled"{$importantface:oh hey there!}"
withfaces(:emphasis => :magenta) do
println(astr)
end
New Sample
importantface = face"emphasis"
astr = styled"{$importantface:oh hey there!}" # Unchanged
withfaces(face"emphasis" => face"magenta") do
println(astr)
end
Namespaces and palettes
Currently, all named faces are put in a shared global dictionary, where packages have to specify the full names set/used. This is done for four reasons:
- Having full names makes mapping user configuration straightforward
- I didn’t think of a nice way to do namespacing
- Nobody suggested a good way of doing namespacing
- Over in Emacs-land, this is what’s done, and everybody adheres strictly to the conventions (it might help that in Emacs, if you try to define a face without your package prefix, you get a warning)
After talking about the design with Cody, I’ve now got a system for both module-level face namespaces, and named collections of faces — palettes. If you hadn’t guessed already, the module namespaces are just palettes with a predictable name.
The public API for this consists of three new macros:
@defpalette!for creating a palette@registerpalette!(placed within__init___) for performing some work at runtime (needed to make user customisation work, along with light/dark theming)@usepalettes!for bringing in faces from certain palettes (thinkusing). Note that this may be used exactly once per module.
Palettes affect how face"<name>" looks up the Face belonging to <name>, letting you skip package prefixes. Instead of :markdown_heading you can call @usepalletes! Markdown and then do face"heading". Dot-access also works (and allows you to reference different faces with the same name from different modules): face"Markdown.heading".
Use of @defpalette!, @usepalettes!, or face"" opts you into a post-Face primacy API, where all faces looked up (by face"" or styled"") must be defined in one of the palettes used.
This all works by having @defpalette! store the definitions in a special named tuple accessed by @usepalettes! and face"". This allows for faces to be looked up at compile (macroexpansion) time instead of runtime. To preserve runtime behaviour (backwards compatibility, the ability to customise faces, etc.) we then intern these faces with @registerpalette! during __init__.
This is the “hybrid” in “hybrid Face interning”. Named Faces are created at compile-time, and passed around/referenced between modules at compile time, and then interned at runtime. This removes the need for addfaces!.
Old Sample
module MyColours
using StyledStrings
const FACES = (
:MyColours_gold => Face(foreground = 0xffd700),
:MyColours_lavender => Face(foreground = 0xe6e6fa),
:MyColours_salmon => Face(foreground = 0xfa8072),
# ...
)
__init__() = foreach(addface!, FACES)
end
# --
module MyTables
using StyledStrings
using MyColours
const FACES = (
:MyTables_header => Face(inherit = :MyColours_lavender, bold = true),
:MyTables_row => Face(inherit = :MyColours_salmon),
:MyTables_rowsep => Face(inherit = :MyColours_gold),
# ...
)
__init__() = foreach(addface!, FACES)
function printtable(header::String, rows::Vector{String}, width::Int)
println(styled"{MyTables_header:$header}\n{MyTables_rowsep:$('-'^width)}")
for row in rows
println(styled"{MyTables_row:$row}")
end
end
end
New Sample
module MyColours
using StyledStrings
@defpalette! begin
gold = Face(foreground = 0xffd700)
lavender = Face(foreground = 0xe6e6fa)
salmon = Face(foreground = 0xfa8072)
# ...
end
__init__() = @registerpalette!
end
# --
module MyTables
using StyledStrings
using MyColours
@usepalettes! MyColours
@defpalette! begin
header = Face(inherit = lavender, bold = true)
row = Face(inherit = salmon)
rowsep = Face(inherit = gold)
end
__init__() = @registerpalette!
function printtable(header::String, rows::Vector{String}, width::Int)
println(styled"{header:$header}\n{rowsep:$('-'^width)}")
for row in rows
println(styled"{row:$row}")
end
end
end
Remapping faces
For backwards compatibility with the hybrid face interning system, we needed the ability to remap faces resolved at compile time to a different face at runtime. After doing this work, it was trivial to introduce a new withfaces-like context wrapper that applies to face construction rather than display.
For example,
mapstr = remapfaces(face"red" => face"blue") do
styled"some {red:important} text"
end
produces styled"some {blue:important} text".
Unsetting faces
In your faces.toml, you can now set attributes to "default" to unset that attribute.
I’m considering renaming the special value string to "unset".
I’m planning on extending this to families/palettes, e.g. Markdown = "unset".
Bonus: adaptive colouring and colour blending
Since I’m doing all this writing, I’ll also mention a separate new feature that I’m quite excited for: the ability to define light/dark face variants and also detect terminal colours and blend faces at runtime to adapt to the colour scheme the user sees.
Light/dark face variants can be defined like so:
@defpalette! begin
region = Face(background=0x636363)
region.light = Face(background=0xaaaaaa)
region.dark = region => Face(background=0x363636)
end
The 16 ANSI colours are also read from the current terminal (when possible), making it possible to adjust styling to match the user’s terminal. This lets us break beyond the limits of 16-bit colour without any compromises, letting us do things like add pretty and customisable (and hence accessible) region highlights.
NB: please pretend you see face"background" instead of :background, these screenshots are a little out of date
Status
As of writing, this has all been implemented, and existing tests are passing. There’s a ~3000 loc PR to StyledStrings I’d love to get a code review on if anybody’s up for the task, and a much smaller ~300 loc WIP PR to the main Julia repo (#60527).
There are some minor issues blocking CI currently (apparently depwarn is an error in Julia CI?), and I’d like to write a bunch of new tests for the new functionality, but it’s all implemented at this point, and seemingly working well 
.
There’s some more I’d write about, but this is already much longer than I anticipated, and it’s much later than I’d like, so I’ll just call it here for now 
I’m very happy to clarify details, and go into depth on aspects of the design (e.g. topologically sorting face dependencies within a palette to allow for order-independent definition).