Normalizing colored output for comparison?


#1

Are there any good options for checking colored output in your tests that work across julia versions?

I keep coming across this issue where I want to validate that some colored output shows up correctly, but the issue is that the exact escape codes used change between Julia versions, so I end up having ugly switches in my test code with differently-escaped versions of the same text.

I’ve thought about writing about something to normalize strings to some kind of canonical representation that can be compared, but I wanted to check to see if there’s a solution already out there.


#2

You can use https://github.com/KristofferC/Crayons.jl


#3

I looked at Crayons.jl a little but it wasn’t clear how it would help my testing case.

As a more concrete example, my DeepDiffs.jl package displays diffs of a variety of datatypes, and can pretty-print them with red/green highlighting:

deepdiffs

Testing the printing functionality is pretty gnarly and each test case needs multiple expected output samples depending on the Julia version. What I really want is a way to test that the output appears a certain way to the user, without being sensitive to exactly the sequence of escape characters used to do the coloring.

I suppose if I used Crayons.jl to do the coloring than the output wouldn’t change with Julia versions, but then it might change with Crayons versions.


#4

additionally - it would be great if I could refactor how I do the printing (which might change the escaping) without needing to regenerate my example output.


#5

That’s what I meant. Output changing from Crayons would be a bug.

Parsing the text into ANSI codes and normalizing sounds like an awful amount of work.


#6

While not exactly addressing your question, I am very familiar with the kind of testing struggle you describe here. maybe a different approach to lighten the burden of testing colored output is using https://github.com/Evizero/ReferenceTests.jl . It supports color and it allows for simple management and updating of the references