OpenCV: issue with function argument type in Julia binding

New to Julia
, ,
1

Hi!
I followed this tutorial to build an OpenCV binding for Julia. Now I am stuck with some functions I do not know how to use. I would like to use the OpenCV.split function whose definition is below in cv_cxx_wrap.jl:

function split(m::InputArray, mv::Array{InputArray, 1})
	return cpp_to_julia(jlopencv_cv_cv_split(julia_to_cpp(m),julia_to_cpp(mv)))
end
split(m::InputArray; mv::Array{InputArray, 1} = (Array{InputArray, 1}())) = split(m, mv)

where InputArray = Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}.
My issue is that I get a no method matching error, I think I do not pass the right argument type as the second argument of split.
Here is a MWE with all the types I tried:

using OpenCV

noisy_img = rand(collect(UInt8, 0:255), (3, 320, 210))

out = Array{OpenCV.InputArray}[]
out = Array{OpenCV.Mat{UInt8}, 1}[]
out = Array{Array{UInt8,1}, 1}[]

out = OpenCV.CxxMat[]
out = AbstractArray{UInt8,3}[]
out = OpenCV.Mat{UInt8}[]
out = Array{UInt8,1}[]

out = Array{Array{UInt8,1}}(undef, 3)
out = Array{OpenCV.InputArray}(undef, 3)
out = Array{OpenCV.Mat{UInt8}}(undef, 3)

out = Array{OpenCV.Mat{UInt8}, 1}()

OpenCV.split(noisy_img, out)  # Split RGB channels

Here is the error I get:

MethodError: no method matching split(::Array{UInt8,3}, ::Array{OpenCV.Mat{UInt8},1})
Closest candidates are:
  split(::Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}; mv) at /home/wahara/build/OpenCV/src/cv_cxx_wrap.jl:2307
  split(::Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, !Matched::Array{Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}},1}) at /home/wahara/build/OpenCV/src/cv_cxx_wrap.jl:2304
top-level scope at functions.jl:138

I am new to Julia and my issues with OpenCV could come from the fact that I do not really know to handle types in Julia. Any help would be appreciated!

2

Welcome! You’ve run into a common confusion, which is the fact that parametric types (including Arrays) in julia are invariant.

Here’s a much simpler example:

julia> function foo(x::Vector{Union{Int, Float64}})
         println(x)
       end
foo (generic function with 1 method)

julia> x = [1, 2, 3]
3-element Vector{Int64}:
 1
 2
 3

julia> foo(x)
ERROR: MethodError: no method matching foo(::Vector{Int64})
Closest candidates are:
  foo(::Vector{Union{Float64, Int64}}) at REPL[1]:1

A Vector{Int} is not a subtype of Vector{Union{Int, Float64}}. That’s because a Vector{Union{Int, Float64}} is its own kind of thing entirely: it’s a container in which each individual element can be an Int or a Float64 separately. A Vector{Int} does not fit that description.

What you probably want is something like “A vector whose element type is some subset of the set {Int, Float64}”. In julia, that’s spelled:

Vector{T} where {T <: Union{Int, Float64}}

or

Vector{<:Union{Int, Float64}}

For example:

julia> function foo2(x::Vector{T}) where {T <: Union{Int, Float64}}
         println(x)
       end
foo2 (generic function with 1 method)

julia> foo2(x)
[1, 2, 3]

julia> function foo3(x::Vector{<:Union{Int, Float64}})
         println(x)
       end
foo3 (generic function with 1 method)

julia> foo3(x)
[1, 2, 3]
2 Likes
3

Thanks a lot for the answer, I did not know that, very useful!
However, I think my issue is different and I did not type the proper definition of InputArray which is

const dtypes = Union{UInt8, Int8, UInt16, Int16, Int32, Float32, Float64}
const InputArray = Union{AbstractArray{T, 3} where {T <: dtypes}, CxxMat}

So I think the problem you mentioned is not the case here.
Actually, I found a way to get a different error (so I think the type issue of my original question is solved) which is the following:

using OpenCV

noisy_img = rand(collect(UInt8, 0:255), (3, 320, 210))
out = OpenCV.InputArray[]
OpenCV.split(noisy_img, out)  # Split RGB channels

And I get

MethodError: no method matching jlopencv_core_Mat_mutable_data(::OpenCV.CxxMatDereferenced)
Closest candidates are:
  jlopencv_core_Mat_mutable_data(!Matched::Union{CxxWrap.CxxWrapCore.SmartPointer{T2}, T2} where T2<:OpenCV.CxxMatAllocated) at /home/wahara/.julia/packages/CxxWrap/OcN1Z/src/CxxWrap.jl:618
cpp_to_julia(::OpenCV.CxxMatDereferenced) at mat_conversion.jl:20
cpp_to_julia at mat_conversion.jl:103 [inlined]
split(::OpenCV.Mat{UInt8}, ::Array{Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}},1}) at cv_cxx_wrap.jl:2305
top-level scope at functions.jl:136

This is a similar issue as in this opened issue about the function findContours of the OpenCV Julia binding.

Alternatively, I am still curious why I get a type error in the following:

using OpenCV

noisy_img = rand(collect(UInt8, 0:255), (3, 320, 210))
out = OpenCV.Mat{UInt8}[]
OpenCV.split(noisy_img, out)  # Split RGB channels

As I have typeof(noisy_img) = Array{UInt8,3} and typeof(out) = Array{OpenCV.Mat{UInt8},1}, I would have thought it would work but I get the following:

MethodError: no method matching split(::Array{UInt8,3}, ::Array{OpenCV.Mat{UInt8},1})
Closest candidates are:
  split(::Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}; mv) at /home/wahara/build/OpenCV/src/cv_cxx_wrap.jl:2307
  split(::Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, !Matched::Array{Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}},1}) at /home/wahara/build/OpenCV/src/cv_cxx_wrap.jl:2304
top-level scope at functions.jl:135
4

Mm, I’m pretty sure it’s still relevant. The error you posted is:

MethodError: no method matching split(::Array{UInt8,3}, ::Array{OpenCV.Mat{UInt8},1})

and the candidate is:

split(::Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, !Matched::Array{Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}},1}) at /home/wahara/build/OpenCV/src/cv_cxx_wrap.jl:2304

from which the relevant part is:

!Matched::Array{Union{...}}

which is exactly what I described above. You’ve used the <: operator to define InputArray, but your split function is still defined using Array{InputArray, 1} as its argument type, whereas it should be Array{<:InputArray, 1} .

1 Like
5

Oh right! This makes a difference indeed. So I redefined the split function as you just wrote with Array{<:InputArray, 1} and now I never have this !Matched issue again. Precisely, I can run the following code where any definition of the out variable does not produce the !Matched issue.

using OpenCV

noisy_img = rand(collect(UInt8, 0:255), (3, 320, 210))

out = OpenCV.CxxMat[]
out = AbstractArray{UInt8,3}[]
out = OpenCV.Mat{UInt8}[]
out = Array{UInt8}[]
out = Array{OpenCV.Mat{UInt8}, 1}()
out = OpenCV.InputArray[]
out = Array{OpenCV.InputArray, 1}()

OpenCV.split(noisy_img, out)

However, in all the cases, I end-up with the other error I mentioned with out = OpenCV.InputArray[] and which is:

MethodError: no method matching jlopencv_core_Mat_mutable_data(::OpenCV.CxxMatDereferenced)
Closest candidates are:
  jlopencv_core_Mat_mutable_data(!Matched::Union{CxxWrap.CxxWrapCore.SmartPointer{T2}, T2} where T2<:OpenCV.CxxMatAllocated) at /home/wahara/.julia/packages/CxxWrap/OcN1Z/src/CxxWrap.jl:618
cpp_to_julia(::OpenCV.CxxMatDereferenced) at mat_conversion.jl:20
cpp_to_julia at mat_conversion.jl:103 [inlined]
split(::Array{UInt8,3}, ::Array{Union{OpenCV.CxxMat, AbstractArray{T,3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}},1}) at cv_cxx_wrap.jl:2305
top-level scope at functions.jl:131
  1. About the definition of split with Array{<:InputArray, 1}. This function is defined in cv_cxx_wrap.jl which is part of the Julia OpenCV Binding. Do you think we should raise the issue in opencv_contrib and suggest your modification?
  2. About the second error (MethodError: no method matching jlopencv_core_Mat_mutable_data(::OpenCV.CxxMatDereferenced)), any thoughts?

Thanks again for your help!

6

I think so. I don’t know how the OpenCV bindings are actually generated, but defining a function whose argument type is Array{Union{<anything>}} is nearly always wrong (and certainly wrong in this case).

Sorry, no–I don’t know anything about the OpenCV Julia stack.

By the way, depending on what you are actually trying to do, you might find the JuliaImages ecosystem easier to use from Julia. Having used opencv from Python in the past, I vastly prefer using the JuliaImages tools and have never wanted to try using opencv from Julia. YMMV, of course.

2 Likes