Is any way to print DataFrames using heat maps as background?

I want to print in various notebooks dataframes setting the background of each cell. This is possible in Python. Here is an example using a heatmap.

Something that looks like:

image

Is some library or code out there to do this? I know how to do this using plotting, I need something different.

3 Likes

If anything this would probably be a job for PrettyTables HTML backend

3 Likes

It should be also possible with text/plain. @Ronis_BR is the best person to comment.

2 Likes

It will be cripple by various consoles constrains on various platforms. I’m more interested by decent printing in various notebooks around like in Python.

Well, the

PrettyTables HTML backend

should work just fine in Jupyter notebooks.

2 Likes

Very good idea! At least in VSCode works great.

using PrettyTables

data = Any[1 false 1.0 0x01
    2 true 2.0 0x02
    3 false 3.0 0x03
    4 true 4.0 0x04
    5 false 5.0 0x05
    6 true 6.0 0x06]

h1 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 1, bold = true, background = 0x220044)
h2 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 2, bold = true, background = 0x440088)
h3 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 3, bold = true, background = 0x6600AA)
h4 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 4, bold = true, background = 0x8800CC)

pretty_table(data, highlighters = (h1, h2, h3, h4))

image

4 Likes

Hi @Dan_Micsa !

You can do this using PrettyTables with the following code:

using Colors
using ColorSchemes
using DataFrames
using PrettyTables

# Function that defines the decoration in each cell.
function colormap_decoration(h, data, i, j)
    max = maximum(eachcol(df)) |> maximum
    min = minimum(eachcol(df)) |> minimum

    # Get the color of the current cell.
    color = get(
        colorschemes[:coolwarm],
        data[i, j],
        (min, max)
    )

    # Convert to hexadecimal.
    hex_color = color |> hex

    # Add to the decoration.
    return HTMLDecoration(background = "#" * hex_color)
end

# Create a ramdom DataFrames.
df = DataFrame(rand(20, 5), :auto)

# Define the highlighter that will add the color to each cell.
hl = HTMLHighlighter(
    (data, i, j) -> true,
    colormap_decoration
)

# Print the table.
pretty_table(df, highlighters = (hl,), backend = :html)

Leading to:

I used the master branch of PrettyTables.jl, but it should work in the released version. Notice that PrettyTables is passing thought a huge rewrite and next version should be some breaking changes.

Side note: this code is not optimized (max and min values are being computed when analyzing each cell), but it works :slight_smile:

8 Likes

I close this case thank you all for answers!

3 Likes

thank you very much for this example.

1 Like

max and min values are being computed when analyzing each cell

I saw that “attrocity” but I was a gentleman. :smile:

2 Likes

:sweat_smile:

1 Like

Another question, if you have a bit of time, how can I pass min and max for color map in your example w/o the ugly globals?

function colormap_decoration(h, data, i, j)
    max = maximum(eachcol(df)) |> maximum
    min = minimum(eachcol(df)) |> minimum
...

The first and easiest is to create a closure:

using Colors
using ColorSchemes
using DataFrames
using PrettyTables

function print_df(df)
    max = maximum(eachcol(df)) |> maximum
    min = minimum(eachcol(df)) |> minimum

    # Function that defines the decoration in each cell.
    function colormap_decoration(h, data, i, j)
        # Get the color of the current cell.
        color = get(
            colorschemes[:coolwarm],
            data[i, j],
            (min, max)
        )

        # Convert to hexadecimal.
        hex_color = color |> hex

        # Add to the decoration.
        return HTMLDecoration(background = "#" * hex_color)
    end

    # Define the highlighter that will add the color to each cell.
    hl = HTMLHighlighter(
        (data, i, j) -> true,
        colormap_decoration
    )

    # Print the table.
    pretty_table(df, highlighters = (hl,), backend = :html)
end

# Create a ramdom DataFrames.
df = DataFrame(rand(20, 5), :auto)
print_df(df)

The second is to define your own highlighter structure:

using Colors
using ColorSchemes
using DataFrames
using PrettyTables

struct MyHighlighter{T}
    f::Function
    fd::Function
    min::T
    max::T
end

# Function that defines the decoration in each cell.
function colormap_decoration(h, data, i, j)
    # Get the color of the current cell.
    color = get(
        colorschemes[:coolwarm],
        data[i, j],
        (h.min, h.max)
    )

    # Convert to hexadecimal.
    hex_color = color |> hex

    # Add to the decoration.
    return HTMLDecoration(background = "#" * hex_color)
end

function print_df(df)
    max = maximum(eachcol(df)) |> maximum
    min = minimum(eachcol(df)) |> minimum

    # Define the highlighter that will add the color to each cell.
    hl = MyHighlighter(
        (data, i, j) -> true,
        colormap_decoration,
        min,
        max
    )

    # Print the table.
    pretty_table(df, highlighters = (hl,), backend = :html)
end

# Create a ramdom DataFrames.
df = DataFrame(rand(20, 5), :auto)
print_df(df)
2 Likes

Thank you. I have over 40y of programming and 2 days of Julia. :slight_smile:

I will play with it. New concepts for me. In C++ I have to write a local class, a function is not possible inside.

1 Like

This is what I get.
Do I need to set something on VS Code?

julia> print_df(df)
<!DOCTYPE html>
<html>
<meta charset="UTF-8">
<style>
  table, td, th {
      border-collapse: collapse;
      font-family: sans-serif;
  }

  td, th {
      border-bottom: 0;
      padding: 4px
  }

I think this is using HTML back end in a console mode. HTML should be used for conversion to HTML or notebooks that can render it graphically.

try this if you using the text like method:

using PrettyTables

data = Any[1 false 1.0 0x01
    2 true 2.0 0x02
    3 false 3.0 0x03
    4 true 4.0 0x04
    5 false 5.0 0x05
    6 true 6.0 0x06]

h1 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 1, bold = true, background = 0x220044)
h2 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 2, bold = true, background = 0x440088)
h3 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 3, bold = true, background = 0x6600AA)
h4 = Highlighter((data, i, j) -> j in (1, 3, 4) && data[i, j] == 4, bold = true, background = 0x8800CC)

#pretty_table(data; highlighters = (h))
pretty_table(data; highlighters = (h1, h2, h3, h4))
#pretty_table(data; highlighters = (h1, h2, h3, h4), backend = Val(:html))

More information here: Home · Pretty Tables

Ronis_BR,

This feels like a bug in the PrettyTables.Crayon. I can’t set both fc and bc. If i set only one of them it works.

using Colors
using ColorSchemes
using DataFrames
using PrettyTables

function pp(df, min_ = nothing, max_ = nothing, isCentered = true, gradientCentered = :coolwarm, gradient = :YlGn_9)    
    if isnothing(min_)
        min_ = minimum(eachcol(df)) |> minimum
    end
    
    if isnothing(max_)
        max_ = maximum(eachcol(df)) |> maximum
    end

    if isCentered && min_ < -0.00001
        min_ = min(min, -max)
        max_ = max(-min, max)
        gradient = gradientCentered
    end

    function colormapDecorationHTML(h, data, i, j)
        color = get(colorschemes[gradient], data[i, j], (min_, max_))
        return HTMLDecoration(color = "#" * (color |> hex), background = convert(Bool, i % 2) ? "#222222" : "#111111")
        #return HTMLDecoration(background = "#" * (color |> hex), color = convert(Bool, i % 2) ? "#222222" : "#111111")
    end

    function colormapDecorationText(h, data, i, j)
        color = get(colorschemes[gradient], data[i, j], (min_, max_))
        color2 = "0x" * (color |> hex)
        color = parse(UInt32, color2)
        #return Crayon(;background = color)
        #return Crayon(;foreground = color)
        #return Crayon(;foreground = 0x888888, background = color)
        return Crayon(;foreground = 0xFFFF00, background = 0x0000FF)#! I can't set both fc and bc !
    end

    # Define the highlighter that will add the color to each cell.
    hlHTML = HTMLHighlighter(f = (data, i, j) -> true, fd = colormapDecorationHTML)
    hlText = Highlighter(f = (data, i, j) -> true, fd = colormapDecorationText)
    
    # Print the table.
    pretty_table(df, highlighters = (hlHTML), backend = :html, formatters = ft_printf("%.4f"))
    pretty_table(df, highlighters = (hlText), formatters = ft_printf("%.4f"))
end

# Create a random DataFrames.
df = DataFrame(rand(10, 5), :auto)
pp(df)

Hi @Dan_Micsa !

Sorry for the delay, I missed the notification…

I run this code and everything seems to be working correctly:

What is the problem in your case?

1 Like

This code looks OK in console but not in VS Code. I believe I disturb you with things that are not from the PrettyTable library.

This code looks OK in console but not in VS Code. I believe i disturb you with thinks that are not from the PrettyTable library.

No problem! Please, feel free to contact me anytime :slight_smile:

It seems to me it is a problem of neither PrettyTables nor Crayons. I believe it might be something related to the console in VSCode. Can you execute the following code in Julia inside the VSCode console please:

julia> println("\e[38;2;255;255;0;48;2;0;0;255mThis must have blue background\e[0m")
julia> println("\e[33;44mThis must have blue background\e[0m")