I’m working on adapting the fast.ai fastbook with a few other people. You can find our work slowly happening via our website or by the Quarto book here.
As part of this, I want to build a little tool to view images with the class printed right above them. To do this, I need to do the following:
Build a white pane that is slightly larger than the underlying image.
Paste the underying image on top of the white pane.
Add text to the top of the white pane.
Anyone have any ideas on how to go about this? I looked at most of the functions in Images.jl and its sattelite packages but I was not able to find any way to compose images or to add text. Happy to be pointed to something if I missed it!
Once you start to wade into composing and annotating images, it’s time to bail out to a plotting library. Makie makes this sort of thing pretty easy:
julia> f = Figure()
julia> for idx in CartesianIndices(imgs)
ax = Axis(f[idx.I...], aspect=DataAspect(),
title="Letter $(rand('a':'z'))")
image!(ax, imgs[idx])
hidedecorations!(ax)
end
If you want to do it without a plotting library, you can simply copy parts of your image matrix around to compose your result.
You can render the text in Cairo, but when doing so I had to overcome a few pitfalls (e.g. for compatibility with Cairo, I just learnt you need to use the Images.RGB24 color type, and documentation for Cairo.jl seems largely absent).
# Suround a bitmap image with a white margin and add text there
using Images
using Cairo
using TestImages
img = testimage("airplaneF16")
margin = 40
title = "U.S. Air Force F-16"
# prepare a white canvas (image size plus margin)
canvas = fill(RGB24(1.0,1.0,1.0), size(img) .+ 2*margin)
# copy image centered onto canvas
canvas[margin+1:margin+size(img,1),
margin+1:margin+size(img,2)] = img
# prepare another canvas for rendering the text into top margin
# (because Cairo draws with X and Y axis fliped relative to Images)
header = fill(RGB24(1.0,1.0,1.0), (size(canvas,2), margin))
c = CairoImageSurface(header)
cr = CairoContext(c)
# render the text
select_font_face(cr, "Sans", Cairo.FONT_SLANT_NORMAL,
Cairo.FONT_WEIGHT_NORMAL)
set_font_size(cr, 20.0)
move_to(cr, (size(header, 1) - textwidth(cr, title)) / 2,
(margin + textheight(cr, title)) / 2)
set_source_rgb(cr,0,0,0) # black
show_text(cr, title)
# copy the x/y flipped top margin back onto main canvas
canvas[axes(header')...] = header'
# Images.save seems to have a bug when receiving Matrix{RGB24}, so convert
Images.save("test.png", RGB{N0f8}.(canvas))
Okay, I like the ideas from @stillyslalom, @mgkuhn and @cormullion. But I think there is a gap to fill here from image related dataset visualization pov.
Makie solution: Makie is a great high level solution but its heavy like quite heavy
Cairo+Luxor: Very flexible and powerful but quite low level for workflow needed by @cpfiffer
I think the solution is likely to eventually come in form of FreeTypeAbstraction.jl(for text render) being part of ImageDraw.jl, to support dataset visualization and such at a slight higher level in form of package extension if MosaicViews.jl is available. Very light and just what we need in a dynamic workflow in my opinion. It could also come from ImageView.jl but right now I think it’s bit heavy.
I second the idea of a Makie or (better due to being lighter?) Cairo/Luxor solution.
I would like a solution to be usable both from a Pluto notebook (easily interpolatable into e.g. HypertextLiteral’s), in a web application (output some web component I can send off as a response from an Oxygen/HTTP method handler) - or just standalone - render something in an SVG, a PDF or (rendered) in a PNG. Nice if it also works in something building on GTK/QML.
… but ImageDraw also seems sufficiently low on dependencies to be an excellent place for implementation. Package extensions for ImageDraw sounds just right - but it would be nice to be able to just stick it an ImageAnnotations AnnotatedImage and get a view back.
Do note however that AnnotatedImage is not up to speed yet - at all: It needs to be transformed into something like AnnotatedImage{TArray<:AbstractArray{T,N}} to support actual image data - both eager- and lazily-loaded data (and not just a path to some image file as it is currently). Cf. Slack thread on lazy loading of images from May 2023.
I think Makie can do everything you want - and if you’re alreadymaking plots with it, you really don’t need anything else. It does take a while to install, though…
) to have the pleasure of actually using Makie … and Plots … and Luxor.)