RGB image operations: addition works, multiplication doesn't?

New to Julia
1

Hi all
I’m new to Julia, working on a script to perform flat field correction on images (as an introductory project, coming from python).

using TestImages, Images, Statistics
img = testimage("mandrill")

Now, looking at the following operations:

  • test = img.*0.5 → works
  • test = img.+img and test = img.-img → works
  • test = img.*img and test = img./img → doesn’t work:
ERROR: MethodError: no method matching *(::RGB{Normed{UInt8,8}}, ::RGB{Normed{UInt8,8}})
Closest candidates are:
  *(::Any, ::Any, ::Any, ::Any...) at operators.jl:529
  *(::Normed, ::AbstractRGB{T}) where T<:Normed at C:\Users\paulyk\.juliapro\JuliaPro_v1.4.1-1\packages\ColorVectorSpace\B9KU5\src\ColorVectorSpace.jl:130
  *(::Real, ::AbstractRGB{T}) where T<:Normed at C:\Users\paulyk\.juliapro\JuliaPro_v1.4.1-1\packages\ColorVectorSpace\B9KU5\src\ColorVectorSpace.jl:126
  ...
Stacktrace:
 [1] _broadcast_getindex_evalf at .\broadcast.jl:631 [inlined]
 [2] _broadcast_getindex at .\broadcast.jl:604 [inlined]
 [3] getindex at .\broadcast.jl:564 [inlined]
 [4] copy at .\broadcast.jl:854 [inlined]
 [5] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Array{RGB{Normed{UInt8,8}},2},Array{RGB{Normed{UInt8,8}},2}}}) at .\broadcast.jl:820
 [6] top-level scope at none:0

Consequently, statistics like test = median(img, dims=(1,2)) also don’t work with a similar error. Any ideas why? I am able to get it working by “converting” to channelview:

img_chanview = channelview(img)
test = img_chanview.*img_chanview
median = median(channelview(img),dims=(2,3))

That works, however this whole process seems to be much slower compared to reading images in as numpy arrays in python and doing similar operations. My RGB files are 36 MP and I need to recombine frames like this. I am working on a laptop Windows 10, intel i7-6600U CPU, the whole process of reading images, converting/operating and writing back to image file took roughly 4.5 sec/image in python but using channelview around 15 sec/image in Julia, so I’m trying to understand how to do it as efficiently as possible.

Best regards
Klaas

1 Like
2

same experience

3

I suspect your problem comes from not having a definition of multiply defined between RGB objects. What is that supposed to mean?
What is RGB1RGB2 supposed to do?
Element wise (R1R2, G1G2, B1B2) ?
Or is it a dot product (R1R2+ G1G2 + B1*B2)
Or something like a vector cross product?

Or think of complex multiplication for the 2 element case: (A+Bim) * (C+Dim) = (AC - BD) + (BC + AD)im ?

Similar for the median problem, it needs a way to be able to sort RGB values. Based on what? Intensity, Hue, Saturation, R, … etc.

Breaking it apart of channels resolves what your doing. I don’t have enough experience with the Image libraries, but you should be able to do this without the channel breakup by custom defining the operators you need.

1 Like
4

About the performance, you could try timing each operation and see where the bottleneck is. First I would try to decouple the load/save operations from the actual Julia processing.
How do you load the images? I observed that FileIO is a bit slower than directly loading with ImageMagick.

5

This seems relevant: Important decisions with respect to color math (please comment) · Issue #126 · JuliaGraphics/ColorVectorSpace.jl · GitHub

/cc @tim.holy, @kimikage

6

This is off-topic, but the median is generally not suitable for fast processing. If you need the true median values and the pixel type is RGB{N0f8}, I think the histogram-based algorithm is fast.

FYI, currently, the multiplication of N0f8 is slow, so I think ColorBlendModes.BlendMultiply is worth considering “for now”. :smile:

7

Thanks all for the input!

The main bottlenecks were actually in converting to channelviews and especially saving back to image.
In the meantime, I took a different approach using ArchGDAL to load images into 3D arrays, that bypasses the bottleneck of channelview very efficiently. I get the point about median being a bottleneck now, I will revisit that later (the whole process of calculating the median (per band) of a 3D array and applying the correction now takes about a second per frame).

@time img_cor=AG.read(AG.read(joinpath(root,file))).*flatmed./flat
→ 1.08 sec per image for my 36 MP files (where flat is the 3D array of the flat field image and flatmed the 3D array with a median value per band, determined beforehand). This is called for each image in a folder. Actually I use this in a function to optimize the execution.

The only problem I still face in that approach is converting the resulting 3D array img_cor back into ArchGDAL dataset (after converting the resulting Float64 back into UInt8) to then write it to a resulting jpeg file, but I should probably address that in another discussion thread.

8

You seem to be using fixed size images (arrays). If so, I think moving pixel values to a pre-allocated array is faster than using views.
I’m not against using the ArchGDAL package, but converting images to/from arrays is very simple.