How to use OpenCV.jl's calibrateCamera correctly?

General Usage
1

Hi,
I’m trying to reproduce :OpenCV: Camera Calibration using OpenCV.jl.
I made some adjustments but could not figure out how to properly pass the arguments to cv.calibrateCamera. 1st and 2nd arguments I passed were Vector{Array{Float32, 3}}s but those were not detected as Vector{AbstractArray{Float32, 3}}. Also, I need the 4th and 5th arguments that we don’t need in Python version.
Could anyone suggest changes to make it work?

using OpenCV
const cv = OpenCV

## file
filename = "calib_radial.jpg"
img = cv.imread(filename, -1)
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# termination criteria
criteria = cv.TermCriteria(cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)

# Arrays to store object points and image points from all the images.
objpoints = Vector{Array{Float32, 3}}(undef, 0) # 3d point in real world space
imgpoints = Vector{Array{Float32, 3}}(undef, 0) # 2d points in image plane.

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = zeros(Float32, 2, 1, 7 * 6)
n = 1
for i in 1:7, j in 1:6
	objp[1, 1, n] = i - 1
	objp[2, 1, n] = j - 1
	n += 1
end

# Find the chess board corners
isValid, corners = cv.findChessboardCorners(gray, cv.Size{Int32}(7, 6))

# If found, add object points, image points (after refining them)
if isValid
	push!(objpoints, objp)
end

corners2 = cv.cornerSubPix(gray, corners, cv.Size{Int32}(11, 11), cv.Size{Int32}(-1, -1), criteria)
push!(imgpoints, corners2)

## camera cal
ret, mtx, dist, rvecs, tvecs = cv.calibrateCamera(objpoints, imgpoints, cv.Size{Int32}(size(gray)[[3, 2]]...))

The error I got.

ERROR: MethodError: no method matching calibrateCamera(::Vector{Array{Float32, 3}}, ::Vector{Array{Float32, 3}}, ::OpenCV.Size{Int32})

Closest candidates are:
  calibrateCamera(::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::OpenCV.Size{Int32}, ::Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, ::Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}; rvecs, tvecs, flags, criteria)
   @ OpenCV C:\Users\---\.julia\artifacts\fb232e63f59cec37f3a904b5decdaa33513a13b1\OpenCV\src\cv_cxx_wrap.jl:1786
  calibrateCamera(::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::OpenCV.Size{Int32}, ::Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, ::Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}, ::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}, ::Int64, ::OpenCV.TermCriteria)
   @ OpenCV C:\Users\---\.julia\artifacts\fb232e63f59cec37f3a904b5decdaa33513a13b1\OpenCV\src\cv_cxx_wrap.jl:1783
1 Like
2

I ran into the exact same issues (described OpenCV function signatures).

Did you get anywhere regarding this?

3

Those are some very strange types indeed. Note for example that

julia> Vector{Array{Float32, 3}} <: Vector{AbstractArray{Float32, 3}}
false

julia> Vector{Array{Float32, 3}} <: Vector{Array{T, 3} where T <: Union{Float32, Float64}}
false

julia> Vector{Array{Float32, 3}} <: Vector{Array{T, 3}} where T <: Union{Float32, Float64}
true

You could, if you really want, convert from Vector{Array{Float32, 3}} to the type in the signature via

objpoints = Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T <: Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}(objpoints)
imgpoints = Vector{Union{OpenCV.CxxMat, AbstractArray{T, 3} where T <: Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}}}(imgpoints)

but I have no idea why the fourth argument (CameraMatrix, cf. OpenCV docs, a 3 \times 3 matrix) would require three axes (Union{OpenCV.CxxMat, AbstractArray{T, 3} where T <: (...)}}) (nor why

julia> OpenCV.Mat <: AbstractArray{T, 2} where T
false

julia> OpenCV.Mat <: AbstractArray{T, 3} where T
true

for that matter.)

I think you’ll have better luck PythonCalling the OpenCV Python bindings.

4

Concretely,

using Pkg
Pkg.add(["PythonCall", "CondaPkg"])
using CondaPkg
CondaPkg.add(["opencv", "numpy"])

using PythonCall
cv = pyimport("cv2")
np = pyimport("numpy")

## file
filename = "calib_radial.jpg"
img = cv.imread(filename, -1)  # (This is a Python object)
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# termination criteria
criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)  # (This is a Julia Tuple, where the first entry is a Python int)

# Python Lists to store object points and image points from all the images.
objpoints = PyList() # 3d points in real world space
imgpoints = PyList() # 2d points in image plane.

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = zeros(Float32, 7 * 6, 3)  # Note: changed compared to original code
let n = 1
	for i in 1:7, j in 1:6
		objp[n, 1] = i - 1
		objp[n, 2] = j - 1
		n += 1
	end
end

# Find the chess board corners
isValid, corners = cv.findChessboardCorners(gray, (7, 6))

# If found, add object points, image points (after refining them)
if pytruth(isValid)  # (or pyconvert(Bool, isValid))
	push!(objpoints, np.asarray(objp))  # objp is Julia, but we need Python for cv, so convert to NumPy
end

corners2 = cv.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
push!(imgpoints, corners2)

## camera cal
ret, mtx, dist, rvecs, tvecs = cv.calibrateCamera(objpoints, imgpoints, reverse(Tuple(gray.shape)), pybuiltins.None, pybuiltins.None)
# Python:
# (82.84610812959355, 
#  array([[1.31550595e+03, 0.00000000e+00, 1.89256059e+02],
#        [0.00000000e+00, 2.72905446e+02, 2.15993621e+02],
#        [0.00000000e+00, 0.00000000e+00, 1.00000000e+00]]),  
#  array([[-9.12214600e+00, -9.69316233e+02, -2.64952158e+00, -1.37557924e+00,  3.36561597e+04]]), 
#  (array([[-1.21794817],
#          [ 0.59696311],
#          [-0.58325645]]),), 
#  (array([[-2.31090673],
#        [ 1.20981955],
#        [22.52187791]]),))

pyconvert(Array, mtx)
# 3×3 Matrix{Float64}:
#  1315.51    0.0    189.256
#     0.0   272.905  215.994
#     0.0     0.0      1.0