Arithmetic operations on an RGB image?

Specific Domains Signal and Image Processing
1

Suppose I have an RGB image, imported with

img = FileIO.load("image.png");

Then I have two questions:

First, in Jupyter, using IJulia, how do I view a small subarray (say 3 x 3) as a set of numeric values? If img was a gray-scale image, then I can view a subarray with

Float64.(img[100:102,200:202])

but this doesn’t work for RGB images.

I can see all the values with dump(img[100:102,200:202]) but that is very inconveniently displayed. I’m looking for outputs such as are shown as examples in Quickstart · JuliaImages In a console these would be given as shown, but IJulia interprets any image or subimage as an array of coloured pixels, and displays them as such. I can use channelview but that gives me three (or four) separate arrays, and I want one array, with each element consisting of the RGB values of the pixels.

Secondly, arithmetic on RGB values. This works:

x = img[100:102,200:202]
x .* 0.5

but these don’t:

round.(x)
x .+ 0.2

There’s clearly something very fundamental I’m misunderstanding about image data structures, but how can you define adding a constant, or rounding, to the RGB values of an image?

Thank you very much,
Alasdair

2

One thing you should keep in mind is that RGB isn’t a linear color space so most math on it will give unexpected results.

3

Yes - I’m quite aware of that. I’m simply experimenting with some dithering algorithms, and I need to do some basic arithmetic on the colour planes. I know there’s a DitherPunk package, but I’m writing my own routines simply so as to understand them better.

I can separate the image using channelview, perform greyscale dithering on each channel and recombine them with colorview, but I was hoping there’d be some way of utilising Julia’s data structures instead.

4

You could poke the display system in IJulia to give you what you want. For example, if tc is

tc = img[1:3, 1:3]

then you could try

Base.show(IOContext(stdout), "text/plain", tc)

to get

3×3 Array{RGB{N0f8},2} with eltype RGB{N0f8}:
 RGB{N0f8}(0.643,0.588,0.278)  RGB{N0f8}(0.247,0.224,0.122)  RGB{N0f8}(0.294,0.169,0.039)
 RGB{N0f8}(0.471,0.49,0.243)   RGB{N0f8}(0.529,0.38,0.129)   RGB{N0f8}(0.216,0.137,0.09)
 RGB{N0f8}(0.388,0.29,0.122)   RGB{N0f8}(0.518,0.463,0.18)   RGB{N0f8}(0.235,0.161,0.141)

but even 3×3 is getting untidy.

Although there isn’t a method to round an RGB{Float64} pixel, you could try adding one:

Base.round(p::RGB{Float64}; kwargs...) = RGB(round(p.r; kwargs...), round(p.g; kwargs...), round(p.b; kwargs...))

which method can then broadcast over an RGB{Float64} image. Seems to work OK:

Screenshot 2023-06-23 at 08.42.07
Screenshot 2023-06-23 at 08.42.071054×1126 295 KB

In your explorations you might encounter GitHub - JuliaGraphics/ColorVectorSpace.jl: Treat colors as if they are n-vectors for the purposes of arithmetic .

5 
julia> using ImageCore

julia> img = rand(RGB{N0f8}, 3, 3)
3×3 Array{RGB{N0f8},2} with eltype RGB{N0f8}:
 RGB{N0f8}(0.922,0.384,0.176)  RGB{N0f8}(0.204,0.741,0.247)  RGB{N0f8}(0.659,0.6,0.373)
 RGB{N0f8}(0.529,0.616,0.486)  RGB{N0f8}(0.427,0.118,0.749)  RGB{N0f8}(0.4,0.353,0.145)
 RGB{N0f8}(0.627,0.953,0.573)  RGB{N0f8}(0.718,0.114,0.353)  RGB{N0f8}(0.329,0.431,0.498)

julia> float.(img)
3×3 Array{RGB{Float32},2} with eltype RGB{Float32}:
 RGB{Float32}(0.921569,0.384314,0.176471)  …  RGB{Float32}(0.658824,0.6,0.372549)
 RGB{Float32}(0.529412,0.615686,0.486275)     RGB{Float32}(0.4,0.352941,0.145098)
 RGB{Float32}(0.627451,0.952941,0.572549)     RGB{Float32}(0.329412,0.431373,0.498039)

julia> img .+ RGB(0.2, 0.2, 0.2)
3×3 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(1.12157,0.584314,0.376471)   …  RGB{Float64}(0.858824,0.8,0.572549)
 RGB{Float64}(0.729412,0.815686,0.686275)     RGB{Float64}(0.6,0.552941,0.345098)
 RGB{Float64}(0.827451,1.15294,0.772549)      RGB{Float64}(0.529412,0.631373,0.698039)

Addition isn’t defined between colors and scalars (except for Gray), but you can always specify each channel. And as @cormullion said, round isn’t defined although we could add it. (Pull requests welcome!)

2 Likes
6

Adding to the list of answers for future reference… it is easy to convert RGB to Gray with Gray.(img). There is also the low-level channelview(img) (it used to be called that) that converts the 2D array of color objects into a 3D arrays of numbers as it is traditional in Matlab and other languages (not recommended).

7

Thank you very much - I’ll look more closely at your round implementation; my Julia programming has been elementary to the extent that I’ve never need kwargs or the splat operator, and indeed I only have the vaguest idea of their use. I imagine here that their use allows for calling round with its various other optional parameters: digits, sigdigits, and base. Very neat - and thank you again.

1 Like
8

Yes - and in fact that is what I have been doing: converting a color image to its color planes with channelview, operating on each plane separately, and then combining the results back into a single image with colorview. But I’m trying to exploit Julia’s data structures to have one single program that will work for images of either grayscale or colour, without needing to determine what type of image it is.

9

That’s entirely possible for a subset of color types. For example, if you want to add 0.2 to each color channel regardless of whether it’s grayscale or RGB you can call img[i,j] + scalarcolor(eltype(img), 0.2) where you’ve defined

scalarcolor(::Type{<:AbstractGray}, x) = x
scalarcolor(::Type{T}, x) where T<:AbstractRGB = T(x, x, x)

But this would make no sense to define for, e.g., HSV, where the hue is in [0, 360] but the saturation and value are in [0, 1]. This is why the colors ecosystem is hesitant about doing naive things unless you make it very clear what you actually want.

2 Likes
10

Thank you very much - yes it makes sense for Julia to have checks against people making unreasonable operations on colorspace components, given the different nature of colorspaces (RGB vs HSV, as you say).