What would be an efficient image type to apply a series of JuliaImages filters?

General Usage Performance
, ,
1

Hey! I am applying a series of filters/operations from JuliaImages to images that are initially of type Array{UInt8,2}. What would be the most efficient image type for this kind of computation?

Should I always convert back to Array{UInt8,2} after application of a filter? Should I rather find a specific type that could be used for all filters without conversion? To me, the issue here is that type conversion makes the overall computation slow.

Filters I use: some keep the image type unchanged, other do change it. I use for instance ImageMorphology.dilate, Images.fastcorners, ImageFiltering.imfilter, Images.canny, Images.imedge. I also use logical operations between pairs of images, such as:

Tp = Array{UInt8,2}
bitwise_and(x::Tp, y::Tp) = (0xff .* (x .& y))
bitwise_or(x::Tp, y::Tp) = (0xff .* (x .| y))
bitwise_wor(x::Tp, y::Tp) = (0xff .* (x .⊻ y))

The operators with their output type are summarized in the table below, assuming that the input (or pair of inputs) is of type Array{UInt8,2}:

Name Kernel (if needed) Output type
ImageFiltering.imfilter Images.Kernel.gaussian Array{Float64,2}
ImageFiltering.imfilter Images.Kernel.Laplacian Array{Int16,2}
ImageFiltering.imfilter Images.Kernel.sobel Array{Float64,2}
ImageFiltering.imfilter [-1 1 -1; 1 0 1; -1 1 -1] Array{Int64,2}
Images.canny Array{Bool,2}
Images.imedge Array{Float64,2}
ImageMorphology.dilate Array{UInt8,2}
ImageMorphology.erode Array{UInt8,2}
bitwise_and Array{UInt8,2}
bitwise_or Array{UInt8,2}
bitwise_and Array{UInt8,2}
label_components Array{Int64,2}

I am also interested in an image type that would be efficient for a subset of the operators above.

Filters ordering: I assume that the operators are applied randomly, there is no specific rule as to which operator should be applied after another one.

2

In JuliaImages we deliberately use N0f8 from FixedPointNumbers to replace UInt8 because otherwise, it’s often very confusing what is white in image processing (1 or 255?).

Converting Array{UInt8, 2} to Array{N0f8, 2} can be done efficiently via reinterpret(N0f8, img_uint8). And the reverse if rawview(img_n0f8)

Many operations like bitwise_and are actually a pointwise operations, and thus it’s better to write the scalar version, e.g.,:

bitwise_and(x::T, y::T) where T<:Number = 0xff * (x & y)

and use broadcasting

C = bitwise_and.(A, B)
# or in-place version
C .= bitwise_and.(A, B)

Didn’t check if the code runs, but I think you can get the idea here.

2 Likes
3

Thanks! About sequentially applying random filters, are you suggesting to first convert to N0f8 image and then never convert any output / input of any filter? This would mean that input type of a filter could be many different things. For instance, applying imfilter with gaussian kernel to an N0f8 image produces an Array{Float64,2} image while applying canny produces an Array{Bool,2} image.

I could not see any incompability so far with the filters I suggested but I wonder if this is a good practice and if this is efficient.

4

In general, allowing the element type to be computed via the algebraic operations makes sense. There’s no penalty for changing output types as long as they are predictable (to Julia) from the input types.

5

This won’t happen if all filters are applied in an in-place manner:

function rand_fill!(X)
    for i in 1:length(X)
        X[i] = rand(Bool)
    end
    return X
end

X = Array{N0f8, 2}(undef, 4, 4)
# X will still of eltype N0f8 because there
# will be an implicit type conversion here
rand_fill!(X)