# Deforming an image according to a matrix of displacements

**URL:** <https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295>\
**Category:** New to Julia\
**Tags:** images, image-processing\
**Created:** [March 29, 2024, 3:00pm UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295 "2024-03-29T15:00:51Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![JonathanH](https://avatars.discourse-cdn.com/v4/letter/j/8dc957/32.png) [@JonathanH](https://discourse.julialang.org/u/JonathanH)\
**Post date:** [March 29, 2024, 3:00pm UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/1 "2024-03-29T15:00:51Z")

</div>

Hi, I’m struggling to do the following in Julia: I have an image and a corresponding matrix of 2D translations at each pixel location. I would like to warp the image according to this transformation and fill with 0 at divergent locations. The obvious thing to do would be to use the warp function from ImageTransformations.jl, but this function appears to only take an SVector as a transformation, not a matrix. Does anyone know of an alternative method? Thanks.

---

<div class="post-metadata">

**Author:** ![empet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/empet/32/221303_2.png) [@empet](https://discourse.julialang.org/u/empet)\
**Post date:** [March 29, 2024, 10:12pm UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/2 "2024-03-29T22:12:00Z")

</div>

Could you give an example of matrix of 2D translations? Is it a mxn- Matrix{Tuple{Int64, Int64}}, where mxn is the image resolution, and M[i, j]=(k,l) means that the pixel (i,j) in the output image is mapped to the position (k,l), in the input image, via a backward transformation? A minimal example is needed to understand exactly how that matrix is defined.

---

<div class="post-metadata">

**Author:** ![JonathanH](https://avatars.discourse-cdn.com/v4/letter/j/8dc957/32.png) [@JonathanH](https://discourse.julialang.org/u/JonathanH)\
**Post date:** [March 30, 2024, 12:33am UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/3 "2024-03-30T00:33:43Z")

</div>

Sorry about that. Yes basically what you’re saying. The translation could be to sub-pixel locations, so interpolation is necessary. I guess a minimal example would be something like:

```julia
u = [-1 2.5; -1 1]
v = [-1 -1; 1 1]
img = [1 2; 3 4]

```

where u is the column translation of each pixel in img and v is the row translation of each pixel in img. I would like to compute a warped image using these pixel-wise translations, with a fill value of zero.

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [March 30, 2024, 2:29am UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/4 "2024-03-30T02:29:30Z")

</div>

```julia
u = [-1 2.5; -1 1]
v = [-1 -1; 1 1]
img = [1 2; 3 4]

using NearestNeighbors
irregular = 1.0*stack(vec(Tuple.(CartesianIndices(img)))) .+ stack(vec.((u,v));dims=1)
irregkdtree = KDTree(irregular)

using OffsetArrays
bbox = (:)(CartesianIndex.(zip(map(((x,y),)->(floor(Int,x-1),ceil(Int,y+1)),extrema.(eachrow(irregular)))...))...)

using DefaultArrays

regular = stack(vec(Tuple.(bbox));dims=2);
idxs, dists = knn(irregkdtree, regular, 2, true);

out = OffsetArray(DefaultArray(0,size(bbox)), bbox)
for (i,I) in enumerate(bbox)
    dists[i][1] > 1.0 && continue
    out[I] = img[idxs[i][1]]
end

```

After this `out` looks like this:

```julia
julia> out
7×6 OffsetArray(::DefaultArray{Int64, 2}, -1:5, -1:4) with eltype Int64 with indices -1:5×-1:4:
 0 1 0 0 0 0
 1 1 1 3 0 0
 0 1 3 3 3 0
 0 0 0 3 4 0
 0 0 2 4 4 4
 0 0 2 0 4 0
 0 0 0 0 0 0

```

Some variations are possible, such as combining several close points (anti-aliasing).

---

<div class="post-metadata">

**Author:** ![empet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/empet/32/221303_2.png) [@empet](https://discourse.julialang.org/u/empet)\
**Post date:** [March 30, 2024, 8:03am UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/5 "2024-03-30T08:03:06Z")

</div>

Since you did’t give an image and the associated u, v vectors I  
derived first, the u, v vectors for a  
nonlinear image transformation, namely through the complex function f(z)=sqrt(z):

```julia
using Images, ImageTransformations, Interpolations    
include("complextransf.jl");
img = load("ai-boy.jpg") 
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.15, B=1.15, C=-1.15, D=1.15));
invtform = get_invtform(obj, invf);
ipixels = []
m, n = size(img)
indices = CartesianIndices((1:m, 1:n))
for pix in indices
    push!(ipixels, invtform([pix.I...]))     
end 
u = getindex.(ipixels, 1)
v = getindex.(ipixels, 2)

### Here ends the definition of vectors u,v and starts the code that performs
### the image transformation based on these vectors
        
m, n = size(img)
outimg = similar(img, size(img));

itp = interpolate(img, BSpline(Linear())) 
for (k, opix) in enumerate(CartesianIndices(outimg))
    if 1<=u[k]<=m && 1<= v[k]<= n
       outimg[opix] = itp(u[k], v[k])
    else
        outimg[opix] = Gray(1.0)
    end    
end
save("outputimage.jpg", outimg)

```

![ai-boy](https://global.discourse-cdn.com/julialang/original/3X/1/0/10bf8a4983a05e65161877db2c62776d2820d2c6.jpeg)

![outimg](https://global.discourse-cdn.com/julialang/original/3X/4/d/4d78929e578e370c6d8d39d4fe743f2cb2ec93b6.jpeg)

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [April 1, 2024, 10:50am UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/6 "2024-04-01T10:50:38Z")

</div>

> [@empet](#):
>
> Since you did’t give an image and the associated u, v vectors I  
> derived first, the u, v vectors for a

I’d like to experiment with these features, but some of the following functions/files are missing in my environment.  
what do I need to add to make these work…?

```julia-auto
include("complextransf.jl");
...
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.15, B=1.15, C=-1.15, D=1.15));
invtform = get_invtform(obj, invf);

```

---

<div class="post-metadata">

**Author:** ![empet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/empet/32/221303_2.png) [@empet](https://discourse.julialang.org/u/empet)\
**Post date:** [April 1, 2024, 12:33pm UTC](https://discourse.julialang.org/t/deforming-an-image-according-to-a-matrix-of-displacements/112295/7 "2024-04-01T12:33:57Z")

</div>

`ComplexImageTransformations.jl` is a small project I started during the pandemic, when I tried to understand how ImageTransformations work. It’s not online.

Mainly, the complex image transformation is defined as follows:  
A mxn-image overlays a grid of the same resolution, on a rectangle [a,b]\times [c,d], from the complex plane.  
A complex function, f, possibly multi-valued (like sqrt, log, asin, etc, but with a uni-valued inverse), transforms the pixels in the input image in the same way it transforms the corresponding grid points from the complex plane.  
The output image, in turn, is superimposed on a mxn - grid of a minimal rectangle [A, B]\times [C,D], containing  
f([a,b]\times[c,d]).

The inverse transformation, `invtform`, is defined by f^{-1}, composed at left and at right with affine transformations that define the correspondence pixel-\> grid point, and conversely, for output, respectively input image.

Image transformation illustrates much better the properties of complex functions, especially multivalued ones, than presenting the effect of the same function on a grid of vertical and horizontal lines, drawn on a given rectangle.

Here is the file `complextransf.jl`[https://gist.github.com/empet/b2447da03d99358fbf90cf2b039a6672](https://gist.github.com/empet/b2447da03d99358fbf90cf2b039a6672).

Update: including `complextransf.jl`, the derivation of the vectors u, v is not necessary. u,v have been required by OP, to avoid using transformations and SVector(s).  
Instead of using u, v, just call the function `complexwarped`:

```julia
include("complextransf.jl");
imgc = load("ai-boy.jpg")
img = Gray.(imgc)
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.1, B=1.1, C=-1.1, D=1.1));
imgw = complexwarped(obj, invf)
save("sqrt-image.jpg", imgw)

```