Type Parameterization of Elements based on `FixedPointNumber.jl` in a Function

General Usage
, , ,
1

I created a function to convert Images.jl’s packed format into a planar format:

using Images;

function ConverImagePlanarForm( mI :: Matrix{RGB{Normed{U, f}}} ) :: Array{U, 3} where{U <: Unsigned, f <: Integer}
    # Converts from Julia Images Packed from to Planar form:
    # R1G1B1R2G2B2R3G3B3 -> R1R2R3G1G2G3B1B2B3

    dataType = FixedPointNumbers.rawtype(eltype(eltype(mI)));
    return permutedims(reinterpret(reshape, dataType, mI), (2, 3, 1));

end

Since Images.jl uses FixedPointNumbers.jl’s types I parameterized both input and output on it.

In FixedPointNumbers.jl’s documentation it is stated that:

the N0f8 type (aliased to Normed{UInt8,8} ) is represented internally by a UInt8…

So the above should work.
I tested it with:

mS = Matrix{RGB{Normed{UInt8, 4}}}([0.1 1.0; 0.9 0.75]);
typeof(mS)

Yields:

Matrix{RGB{N4f4}} (alias for Array{RGB{Normed{UInt8, 4}}, 2})

Which is great, since it means that U will be parameterized as UInt8 and f is indeed an integer.

Yet:

julia> ConverImagePlanarForm(mS)
ERROR: MethodError: no method matching ConverImagePlanarForm(::Matrix{RGB{N4f4}})
Closest candidates are:
  ConverImagePlanarForm(::Array{RGB{Normed{U, f}}, 2}) where {U<:Unsigned, f<:Integer}

I don’t understand why it doesn’t catch the method defined as they should match.
I have a feeling that UInt8’s type is something different.

Is there a correct way to achieve what I want?

2

Why not let Julia do the type inference, like so:

function ConverImagePlanarForm( mI )
    dataType = FixedPointNumbers.rawtype(eltype(eltype(mI)));
    return permutedims(reinterpret(reshape, dataType, mI), (2, 3, 1));
end
3

@Dan , Mostly because I’d like to understand how to use it correctly in this non trivial (To me at least) case.

Moreover, setting the types here is a good way to ensure the data type in the input matches the assumptions in the code.

In this case, inferring U from the type system might also make dataType = FixedPointNumbers.rawtype(eltype(eltype(mI))); redundant :-).

4 
permutedims(reinterpret(reshape, eltype(eltype(mS)), mS), (2,3,1))

also works to reshape image, so no need to delve into Fixed point representation. Always best to keep operation as generic as possible, to be more adaptable to other representations.

5

@Dan , This won’t yield UInt8 for the case the image it of type Matrix{RGB{N0f8}}. It will yield an array of N0f8. Anyhow, my question was about understanding the correct way to use the type system in this case.

6

I managed to make it work with:

function ConverImagePlanarForm( mI :: Matrix{<: RGB{<: Normed{U}}} ) :: Array{U, 3} where{U <: Unsigned}
    # Converts from Julia Images Packed from to Planar form:
    # R1G1B1R2G2B2R3G3B3 -> R1R2R3G1G2G3B1B2B3

    return permutedims(reinterpret(reshape, U, mI), (2, 3, 1));

end

Yet it was by trial and error. Could anyone assist me why this works?

9

OK, I think I know what has bothered you, f <: Integer is not satisfied, as it is a Type condition, not an isa condition. So 4 <: Integer is false and the call didn’t match the function definition.

The 4 or f is called a free variable in this parametric type, and cannot be matched by the where clause. The function can restrict it with an @assert f == 4 statement inside the function. The way to fully specify the where clause would be:

function ConvertImagePlanarForm( 
  mI :: Matrix{RGB{Normed{U, f}}} ) :: Array{U, 3} where{U <: Unsigned, f}
           @assert f == 4  # not really necessary, but possible with type system
           dataType = FixedPointNumbers.rawtype(eltype(eltype(mI)));
           return permutedims(reinterpret(reshape, dataType, mI), (2, 3, 1));
end

Another better (and slower) way to achieve same thing would be:

ConvertImagePlanarForm(mI) = 
  [getfield(e,f) for e in mI, f in ColorTypes.colorfields(eltype(mI))]

This avoids reinterpret (it does the permutedims organically).

This following version, is even faster than reinterpret version, and also forces the correct semantics of rgb order regardless of underlying type:

function ConvertImagePlanarForm(mI)
    res = Array{eltype(eltype(mI))}(undef, size(mI)..., 3)
    res[:,:,1] .= (px->getfield(px,:r)).(mI)
    res[:,:,2] .= (px->getfield(px,:g)).(mI)
    res[:,:,3] .= (px->getfield(px,:b)).(mI)
    return res
end

It does only the single minimum allocation, and gets the :+1: at the moment.

1 Like
10

Indeed the condition on f makes things fail.
What I don’t understand is why this holds:

julia> Matrix{RGB{N0f8}} <: Matrix{<: RGB{<: Normed{U}}} where{U <: Unsigned}
true

While this fails:

Matrix{RGB{N0f8}} <: Matrix{RGB{<: Normed{U}}} where{U <: Unsigned}
false

and

julia> Matrix{RGB{N0f8}} <: Matrix{RGB{Normed{U}}} where{U <: Unsigned}
false

Or even:

julia> Type{Matrix{RGB{N0f8}}} isa Matrix{RGB{Normed{UInt8, 8}}}
false

Thanks for the solution, but I still think the type is needed to make sure the correct type of the image is the input to the function. It should work only for images of Images.jl which are based on Normed data type.

11

The type system can get confusing at times. So thanks for the question which got me to learn a few things. Good sources I’ve found:
https://docs.julialang.org/en/v1/manual/types/#man-typet-type
and

As for the specific cases:

Here it is important to note Julia abstract types are invariant, which means:

julia> Vector{Float64} <: Vector{Real}                                                                                   
false

even though Float64 <: Real. In this example:

N0f8 <: Normed{U} where {U}    # true. but...
N0f8 == Normed{U} where {U}    # false

which is the same situation as earlier with Float64 and Real.

Next,

Omitting a parameter T is the same as adding a {T} to where clause, and thus:

( Normed{U} where {U} ) == ( Normed{U,T} where {U,T} )   # true

The above can be rewritten as:

Matrix{RGB{N0f8}} <: Matrix{RGB{Normed{U,T}}} where{U <: Unsigned,T}

N0f8 is an alias to Normed{UInt8, 8} and so we can rewrite:

Matrix{RGB{Normed{UInt, 8}} <: Matrix{RGB{Normed{U,T}}} where{U <: Unsigned,T}

Again by invariance this is false.

The next case:

is false because the type of value before isa is not the type after isa, in fact it is the other way around and:

julia> Matrix{RGB{N0f8}} isa Type{Matrix{RGB{Normed{UInt8, 8}}}}                                                         
true

In general T isa Type{T} and Type{T} is the type-of-the-type. This is useful to force parameters of parametric functions to be types (see the references at the top).

Lastly, getting back to first case:

The {<: type} where {...} notation is syntactic-sugar for {T} where {..., T <: type}. Armed with this knowledge rewriting the second term gives:

Matrix{<:RGB{<:Normed{U}}} where {U<:Unsigned} == 
  Matrix{S} where {U <: Unsigned, S <: RGB{T} where {T<:Normed{U}}}

Now we can more easily see that:

Matrix{RGB{N0f8}} <: 
  Matrix{S} where {U <: Unsigned, S <: RGB{T} where {T<:Normed{U}}}

since S = RGB{N0f8} and
indeed S <: RGB{T} where {T<:Normed{U}}
because RGB{N0f8} <: RGB{T} where {T<:Normed{U}}
with T = N0f8 because N0f8 <: Normed{U}
by N0f8 = Normed{UInt8, 8}
and Normed{UInt8, 8} <: Normed{U}
with U = UInt8
and Normed{UInt8, 8} <: Normed{UInt8}
by missing parameters considered “free” so Anytype{S,T} <: Anytype{S}.

It’s all clear as mud… hopefully this discussion diluted this mud a bit :stuck_out_tongue_winking_eye:

1 Like
12

@Dan , Thank you for the effort.
Indeed you made things a bit clearer.

So whenever the type is someType{otherType} we must use <: and not the actual types. Is that correct?

13

Basically yes, if otherType is not a concrete type. Because the only subtypes in this case will be abstract.

1 Like