PrettyTables diff digits highlight?

[LaurentPlagne]

Hi,

First, kudos again to @Ronis_BR for the super useful PrettyTables.jl package !

I was wondering if it could be possible to highlight diff digits between two columns with PrettyTables.jl like

[rafael.guerra]

Are you only interested in a PrettyTables solution?

[LaurentPlagne]

Hi Rafael,

I first wanted to check that it was not yet possible to achieve this with PrettyTables.jl and maybe suggest that it could be a nice extension for the PT highlighters.

I would like to know if other PT users find this potential feature interesting.

Meanwhile I guess I can manage to post-process LateX Pretty tables.jl outputs to obtain what I want. I think that a direct PR in PT would be more difficult for me to achieve.

[Ronis_BR]

Hi @LaurentPlagne !

Currently, the highlighter works only on the entire cell. However, I think we can do something like this using a custom cell called AnsiTextCell. However, all the heavy lift (finding the diff and decorating the string) will be external to PrettyTables.jl,

I will try to do a MWE tomorrow.

[LaurentPlagne]

Thank you @Ronis_BR.
This is cool that the PrettyTable.jl’s internals already allow for this.

@Ronis_BR

I will try to do a MWE tomorrow.

No rush

[ffevotte]

Not sure whether there is a better way to achieve this, but the following seems to “work”:

n = 10
x = rand(n)
y = x .+ 2e-3 .* (rand(n) .- 0.5)

y[1] = x[1]                      # test identical numbers
x[end], y[end] = (0.1, 0.99999)  # what to do here?

using PrettyTables, Crayons, Printf

function text_diff(a, b)
    a_str = @sprintf "%.5f" a
    b_str = @sprintf "%.5f" b
    i = findfirst(i->a_str[i]!=b_str[i], 1:lastindex(a_str))
    if !isnothing(i)
        common = SubString(b_str,1,i-1)
        different = SubString(b_str, i)
        red = crayon"red bold"
        b_str = AnsiTextCell("$common$red$different")
    end
    [a_str b_str]
end

table = reduce(vcat, text_diff(x[i], y[i]) for i in eachindex(x))
pretty_table(table)

---

Note that “different digits” is actually interpreted here as “characters that are different in the decimal representation of the numbers”. But this is probably not really what you want, as I tried to illustrate in the example of the last row.

A more formal definition of the number of common (decimal) digits between two numbers a and b would involve computing a quantity like \text{floor}\left(-\text{log}_{10}\left(\frac{|b-a|}{|a|}\right)\right). But then it gets more difficult to clearly represent it using colored digits in the decimal representation of the numbers.

[LaurentPlagne]

argh, I did not think about this …

I guess that the solution would be to only colorize different significant digits (no coloring at all the last line in this cases) since the aim is to highlight the numerical difference magnitude.

Thank you anyway for the solution to this request !

[Ronis_BR]

Hi @ffevotte !

Your solution is exactly what I had in mind. Thanks!

[ffevotte]

@LaurentPlagne

I guess that the solution would be to only colorize different significant digits (no coloring at all the last line in this cases) since the aim is to highlight the numerical difference magnitude.

In that case I would recommend using something other than a textual difference. My preference goes to the work by D. Stott Parker, especially the definition of the “number of (absolute) significant digits” introduced in his “\frac 1 2 ulsp algorithm”:

using PrettyTables, Crayons, Printf

function highlight_different_digits(a, b)
    # Stott Parker's definition of the "number of (absolute)
    # significant digits" of `b` seen as an approximation of `a`.
    s = -log10(2*abs(b-a)/10.0^(1+floor(log10(abs(a)))))

    j = floor(Int, min(s, 100))
    j > 0 && (j+=1) # account for the dot in 2nd place in scientific notation

    a_str = @sprintf "%.10e" a
    b_str = @sprintf "%.10e" b
    j = clamp(j, 0, lastindex(b_str))

    common    = SubString(b_str, 1, j)
    different = SubString(b_str, j+1)
    red = crayon"red bold"
    b_str = AnsiTextCell("$common$red$different")
    [a_str, b_str]
end

# Random test data
n = 10
x = rand(n)
y = x .+ 2e-3 .* (rand(n) .- 0.5)

# Corner cases
y[1] = x[1]
x[end], y[end] = (0.1, 0.099998)

table = reduce(vcat, permutedims(highlight_different_digits(x[i], y[i])) for i in eachindex(x))
pretty_table(table, header=["Reference value", "Approximation"])

---

Ref: Douglas Stott Parker, “Monte Carlo Arithmetic : exploiting randomness in floating-point arithmetic”, 1997 (pages 19-20).

[LaurentPlagne]

Super cool and mathematically grounded ! Big Thx !
I will avoid coloring the exponent part e-xy but appart from that it is perfect for my need !

Additional question : what if a is zero ?