Drawing text over pixels of an image

Specific Domains Visualization
1

Hello Julia plotting experts, I want to annotate images by drawing text over the pixels of an image. This is to generate a lossless video where each frame shows an animated grid of tiles, each output by a different super-resolution algorithms (using VideoIO.jl). The pixels must remain accurate, i.e. they shouldn’t get blurred, change dimensions, subpixel-shift, etc. Ideally I’d like this to run my script as headless command-line tool so I’d like to avoid dealing with OpenGL windows, and/or saving temporary images to disk.

I’m easily able to load the uncompressed images, extract the section that will become tiles, and composite them through Array manipulation, then feed them frame-by-frame to VIdeoIO. The issue I have is with adding text labels above or ideally overlaid on top of each tile, without messing up the tiles’ pixels.

The closest I’ve gotten to a solution so far is:

using Plots

#... load the image tile into `img`...  e.g. 320x320 pixels

plot(section, title="bla", 
		 #xformatter=x->2x, yformatter=y->2y, 
		 axis=nothing, 
		 border=:none, fmt=:png, 
                 size=(320+40*2, 320+40*2), # try to guess how much padding is still left on the edges
		 xaxis=false, ticks=false,
	     padding=(0, 0))

This approach suffers from many issues:

  1. The tile pixels aren’t aligned to the rendered pixels. There is a subpixel shift (half a pixel I guess).
  2. There remains a white border in the resulting png
  3. I couldn’t find an easy way to extract a pixel array from the png image, without saving it to disk and reloading it.
  4. I couldn’t find an easy way to reposition the title within the tile, e.g. bottom-left corner within the tile. But I could work-around this if I get an array.

Worst case, if problem 3 is resolved, I could use Plots.jl as an overly-complex text renderer, and manually composite the title over the tiles’ array. But surely there’s a more elegant way. And I sure would like to take advantage of Plots.jl other features down the line, e.g. to overlay graphs over the tile.

Thanks a lot!

Christian
PS: I confirmed gr is the default back-end. I don’t mind using other back-ends if that helps.

2

There is not an example in Cairo.jl/Samples.md at master · JuliaGraphics/Cairo.jl · GitHub (closest is the sample clip image) but 1:1 pixel composition can be done in Cairo - you need to do some math to keep scaling exact. Text rendering to bitmaps also.

Home · JuliaImages should show something on input and output.
Both libraries would support in-memory operation only (and no OpenGL).

2 Likes
3

Thanks Andreas! I was running out of time and since I couldn’t come up with any simple solution, I ended up with this hacky method, where I save the transparent plot to png to disk, and load it back. I also trim excessive pixels since the size of the resulting PNG is usually inconsistent. Maybe this can help others or someone can recommend a simplification.

using Images, LRUCache, Plots

"""Alpha blend B over A, using B's alpha (opacity) channel.

    A is at least RGB, B is RGBA.
"""
function alpha_blend(A, B)
    @assert size(A) <= size(B) # B assumed to be of size >= to A's.
    height,width = size(A)
    A_r,A_g,A_b   = eachslice(Float64.(channelview(A)), dims=1)
    B_r,B_g,B_b,α = eachslice(Float64.(channelview(B[1:height,1:width])), dims=1)
    C_r = A_r .* (1 .- α) .+ (B_r .* α)
    C_g = A_g .* (1 .- α) .+ (B_g .* α)
    C_b = A_b .* (1 .- α) .+ (B_b .* α)
    RGB.(C_r, C_g, C_b)
end

""" Plot a transparent PNG image of the specified size, with the specified title.

    Can later be alpha-blended over an image of the specified size.

    Args:
        size: (width,height) in pixels
"""
function title_overlay_img(title::String, size::Tuple{Int,Int}, font_color::Any; 
                           fn="temp_title_overlay_img.png")
    plot(title=title, titlefont=font(12,"times",font_color),
         background=:transparent,
         axis=nothing, 
         border=:none, fmt=:png, 
         size=(size[1]+1, size[2]+1), # the size isn't consistently respected, usually by +/- 1.
         xaxis=false, ticks=false,
         padding=(0, 0))
    savefig(fn)
    load(fn)[1:size[1],1:size[2]] # trim excessive pixels
end;

# const
lru_title_overlay_img = LRU{Tuple{String, Tuple{Int,Int}, Symbol}, 
                                  Array{RGBA{Float64},2}}(maxsize=16)
function cached_title_overlay_img(title::String, size::Tuple{Int,Int}, font_color=:white)
    get!(lru_title_overlay_img, (title, size, font_color)) do
        title_overlay_img(title, size, font_color)
    end
end