Help converting a transformation matrix

General Usage
,
1

For lack of a better alternative, I’m using Matlab’s image calibration toolbox. One of the things I’m after is the transformation matrix Matlab’s fitgeotrans outputs:

tform = fitgeotrans(imagePoints, worldPoints, 'projective');

In tform is a 3x3 matirx, tform.T, which I think is a transformation matrix. I’d like to use this T in Julia.

I “found out” that applying Matlab’s transformPointsForward with tform:

[worldPoints_X, worldPoints_Y] = transformPointsForward(tform, imagePoints_X, imagePoints_Y);

has the exact same results when using T in Julia:

worldPoint_XY = [imagePoint_X imagePoint_Y 1.0]*T

where imagePoint_X and imagePoint_Y is the image coordinate and worldPoint_XY now contains the world coordinate.

My question is: How can I convert this tform to a transformation in CoordinateTransformations.jl?

One of my goals is to use the CoordinateTransformations.Transformation with ImageTransformations.warp as well as transforming image coordinates to world coordinates.

Thanks in advance!

2

Hi, looking at Matlab’s documentation:

Affine2D:
Forward 2-D affine transformation, specified as a nonsingular 3-by-3 numeric matrix.
The matrix T uses the convention:
[x y 1] = [u v 1] * T
where T has the form:
[a b 0; c d 0; e f 1];

T is composed of a 2D rotation:
rot = LinearMap(T[1:2,1:2]')
and a translation:
trans = Translation(T[3,1:2])
such as:
tform = trans ∘ rot

Note: T[1:2,1:2]' is used because CoordinateTransformations uses the other convention:
[x,y] = T*[u,v]

tform can now be applied to imagePoints:
worldPoints_XY = [tform(p) for p in eachrow(imagePoints)]

Note: for better performance, it is recommended to use StaticArrays for representing these small vectors.

1 Like
3

Awesome! I’ll try it out! Thank you.

4

I’m running into problems because the third column of T is not exactly 0,0,1:

julia> tform
3×3 Array{Float64,2}:
    0.371599    -0.0328841  0.000191534
    0.0463963    0.368681   0.000134089
 -285.262      -96.424      1.0

This is admittedly odd and “out of reach” because it’s coming from Matlab’s fitgeotrans, but I thought it’s worth asking… Any idea what I can do about it?

5

I’ve tracked down Matlab’s implementation for these transformation matrices. Here it is:

imagePoints = rand(10,2);
worldPoints = rand(10,2);
tform = fitgeotrans(imagePoints, worldPoints, 'projective'); % create the transformation matrix

Uorg = [1, 2; 3, 4]; % some coordinates we want to transform
V1 = transformPointsForward(tform, Uorg) % Matlab's own function for transforming these coordinates

%% the following is what's happening in the `transformPointsForward` function:
U = padarray(Uorg,[0 1],1,'post') # add a third column of ones
X = U*tform.T % linear algebra
m = repmat(X(:,3),[1 2]) % these are residuals we need to normalize with..?
X(:,1:2) = X(:,1:2)./m % normalizing? 
V2 = X(:,1:2) % this is equal to `V1`

I think I failed to recognize T (from before) as a projective transformation matrix (and not an affine one). It is therefore not enough to include just translation and rotation (and within rotation, scale too). We need another term I think? I strongly suspect that the answer is in https://github.com/JuliaGeometry/CoordinateTransformations.jl#perspective-transformations. I’m just not sure how exactly I can decompose this T to a PerspectiveMap() ∘ inv(AffineMap(...)).

Appreciate any input you might have!

6

OK!
I got it figured out, it’s of course much simpler than I thought. My original goal was to convert matlab’s transformation matrix to a CoordinateTransformations one. I can do that without decomposing them into translation, rotation, scale, and shear (or what not). I can simply do:

M = LinearMap(SMatrix{3,3, Float64}(T'))
tform = PerspectiveMap() ∘ M ∘ push1

where T is matlab’s transformation matrix (and therefore needs a transpose, '), and push1 is simply:

push1(x) = CoordinateTransformations.push(x, 1)

All of this is from Tim Holy’s answer on SO to a related question (Perspective warp an image in Julia - Stack Overflow). Thank you @tim.holy and @touste !

So now, the coordinate transformation in Julia , tform from above, generates the same results as Matlab’s T when applied to 2D coordinates. Yay! :fireworks:

But I’m still stuck on one irritating detail: When I apply this projective transformation matrix to an image with ImageTransformations.warp I run into problems related to the fact that the transformation tform is not invertible. The only way I managed to solve this (I think) is:

wimg = warp(img, itform, ImageTransformations.autorange(img, tform))

where itform is simply:

itform = PerspectiveMap() ∘ inv(M) ∘ push1

and to plot it (with Makie):

image!(ax, ImageTransformations.autorange(img, tform)..., parent(wimg))

where ax is a axis from MakieLayout.

This seems some what convoluted, which is totally fine, but I’m just worried I messed it up in some way…

I’ll also flag for future readers the issue of the convention about where the origin is in an image. In Matlab it’s in the top left corner of the image, so when you detect the checkerboard corners with detectCheckerboardPoints you get xy coordinates that are plottable (they plot correctly on the image) but cannot be used as indices (you will not find the corner in the x column and y row, rather ImageHeight - y). This complicates things even further… I think @tim.holy solved this ambiguity in Julia with ImageAxes.jl.

1 Like