# Help converting a transformation matrix

For lack of a better alternative, I’m using Matlab’s image calibration toolbox. One of the things I’m after is the transformation matrix Matlab’s `fitgeotrans` outputs:

``````tform = fitgeotrans(imagePoints, worldPoints, 'projective');
``````

In `tform` is a 3x3 matirx, `tform.T`, which I think is a transformation matrix. I’d like to use this `T` in Julia.

I “found out” that applying Matlab’s `transformPointsForward` with `tform`:

``````[worldPoints_X, worldPoints_Y] = transformPointsForward(tform, imagePoints_X, imagePoints_Y);
``````

has the exact same results when using `T` in Julia:

``````worldPoint_XY = [imagePoint_X imagePoint_Y 1.0]*T
``````

where `imagePoint_X` and `imagePoint_Y` is the image coordinate and `worldPoint_XY` now contains the world coordinate.

My question is: How can I convert this `tform` to a transformation in `CoordinateTransformations.jl`?

One of my goals is to use the `CoordinateTransformations.Transformation` with `ImageTransformations.warp` as well as transforming image coordinates to world coordinates.

Hi, looking at Matlab’s documentation:

Affine2D:
Forward 2-D affine transformation, specified as a nonsingular 3-by-3 numeric matrix.
The matrix `T` uses the convention:
`[x y 1] = [u v 1] * T`
where `T` has the form:
`[a b 0; c d 0; e f 1];`

`T` is composed of a 2D rotation:
`rot = LinearMap(T[1:2,1:2]')`
and a translation:
`trans = Translation(T[3,1:2])`
such as:
`tform = trans ∘ rot`

Note: `T[1:2,1:2]'` is used because CoordinateTransformations uses the other convention:
`[x,y] = T*[u,v]`

`tform` can now be applied to imagePoints:
`worldPoints_XY = [tform(p) for p in eachrow(imagePoints)]`

Note: for better performance, it is recommended to use StaticArrays for representing these small vectors.

1 Like

Awesome! I’ll try it out! Thank you.

I’m running into problems because the third column of `T` is not exactly `0,0,1`:

``````julia> tform
3×3 Array{Float64,2}:
0.371599    -0.0328841  0.000191534
0.0463963    0.368681   0.000134089
-285.262      -96.424      1.0
``````

This is admittedly odd and “out of reach” because it’s coming from Matlab’s `fitgeotrans`, but I thought it’s worth asking… Any idea what I can do about it?

I’ve tracked down Matlab’s implementation for these transformation matrices. Here it is:

``````imagePoints = rand(10,2);
worldPoints = rand(10,2);
tform = fitgeotrans(imagePoints, worldPoints, 'projective'); % create the transformation matrix

Uorg = [1, 2; 3, 4]; % some coordinates we want to transform
V1 = transformPointsForward(tform, Uorg) % Matlab's own function for transforming these coordinates

%% the following is what's happening in the `transformPointsForward` function:
X = U*tform.T % linear algebra
m = repmat(X(:,3),[1 2]) % these are residuals we need to normalize with..?
X(:,1:2) = X(:,1:2)./m % normalizing?
V2 = X(:,1:2) % this is equal to `V1`
``````

I think I failed to recognize `T` (from before) as a projective transformation matrix (and not an affine one). It is therefore not enough to include just translation and rotation (and within rotation, scale too). We need another term I think? I strongly suspect that the answer is in https://github.com/JuliaGeometry/CoordinateTransformations.jl#perspective-transformations. I’m just not sure how exactly I can decompose this `T` to a `PerspectiveMap() ∘ inv(AffineMap(...))`.

Appreciate any input you might have!

OK!
I got it figured out, it’s of course much simpler than I thought. My original goal was to convert matlab’s transformation matrix to a `CoordinateTransformations` one. I can do that without decomposing them into translation, rotation, scale, and shear (or what not). I can simply do:

``````M = LinearMap(SMatrix{3,3, Float64}(T'))
tform = PerspectiveMap() ∘ M ∘ push1
``````

where `T` is matlab’s transformation matrix (and therefore needs a transpose, `'`), and `push1` is simply:

``````push1(x) = CoordinateTransformations.push(x, 1)
``````

All of this is from Tim Holy’s answer on SO to a related question (https://stackoverflow.com/a/45782741/2261957). Thank you @tim.holy and @touste !

So now, the coordinate transformation in Julia , `tform` from above, generates the same results as Matlab’s `T` when applied to 2D coordinates. Yay!

But I’m still stuck on one irritating detail: When I apply this projective transformation matrix to an image with `ImageTransformations.warp` I run into problems related to the fact that the transformation `tform` is not invertible. The only way I managed to solve this (I think) is:

``````wimg = warp(img, itform, ImageTransformations.autorange(img, tform))
``````

where `itform` is simply:

``````itform = PerspectiveMap() ∘ inv(M) ∘ push1
``````

and to plot it (with Makie):

``````image!(ax, ImageTransformations.autorange(img, tform)..., parent(wimg))
``````

where `ax` is a axis from `MakieLayout`.

This seems some what convoluted, which is totally fine, but I’m just worried I messed it up in some way…

I’ll also flag for future readers the issue of the convention about where the origin is in an image. In Matlab it’s in the top left corner of the image, so when you detect the checkerboard corners with `detectCheckerboardPoints` you get `xy` coordinates that are plottable (they plot correctly on the image) but cannot be used as indices (you will not find the corner in the `x` column and `y` row, rather `ImageHeight - y`). This complicates things even further… I think @tim.holy solved this ambiguity in Julia with ImageAxes.jl.

1 Like