Hi, I’m struggling to do the following in Julia: I have an image and a corresponding matrix of 2D translations at each pixel location. I would like to warp the image according to this transformation and fill with 0 at divergent locations. The obvious thing to do would be to use the warp function from ImageTransformations.jl, but this function appears to only take an SVector as a transformation, not a matrix. Does anyone know of an alternative method? Thanks.
Could you give an example of matrix of 2D translations? Is it a mxn- Matrix{Tuple{Int64, Int64}}, where mxn is the image resolution, and M[i, j]=(k,l) means that the pixel (i,j) in the output image is mapped to the position (k,l), in the input image, via a backward transformation? A minimal example is needed to understand exactly how that matrix is defined.
Sorry about that. Yes basically what you’re saying. The translation could be to sub-pixel locations, so interpolation is necessary. I guess a minimal example would be something like:
u = [-1 2.5; -1 1]
v = [-1 -1; 1 1]
img = [1 2; 3 4]
where u is the column translation of each pixel in img and v is the row translation of each pixel in img. I would like to compute a warped image using these pixel-wise translations, with a fill value of zero.
u = [-1 2.5; -1 1]
v = [-1 -1; 1 1]
img = [1 2; 3 4]
using NearestNeighbors
irregular = 1.0*stack(vec(Tuple.(CartesianIndices(img)))) .+ stack(vec.((u,v));dims=1)
irregkdtree = KDTree(irregular)
using OffsetArrays
bbox = (:)(CartesianIndex.(zip(map(((x,y),)->(floor(Int,x-1),ceil(Int,y+1)),extrema.(eachrow(irregular)))...))...)
using DefaultArrays
regular = stack(vec(Tuple.(bbox));dims=2);
idxs, dists = knn(irregkdtree, regular, 2, true);
out = OffsetArray(DefaultArray(0,size(bbox)), bbox)
for (i,I) in enumerate(bbox)
dists[i][1] > 1.0 && continue
out[I] = img[idxs[i][1]]
end
After this out looks like this:
julia> out
7×6 OffsetArray(::DefaultArray{Int64, 2}, -1:5, -1:4) with eltype Int64 with indices -1:5×-1:4:
0 1 0 0 0 0
1 1 1 3 0 0
0 1 3 3 3 0
0 0 0 3 4 0
0 0 2 4 4 4
0 0 2 0 4 0
0 0 0 0 0 0
Some variations are possible, such as combining several close points (anti-aliasing).
1 Like
Since you did’t give an image and the associated u, v vectors I
derived first, the u, v vectors for a
nonlinear image transformation, namely through the complex function f(z)=sqrt(z):
using Images, ImageTransformations, Interpolations
include("complextransf.jl");
img = load("ai-boy.jpg")
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.15, B=1.15, C=-1.15, D=1.15));
invtform = get_invtform(obj, invf);
ipixels = []
m, n = size(img)
indices = CartesianIndices((1:m, 1:n))
for pix in indices
push!(ipixels, invtform([pix.I...]))
end
u = getindex.(ipixels, 1)
v = getindex.(ipixels, 2)
### Here ends the definition of vectors u,v and starts the code that performs
### the image transformation based on these vectors
m, n = size(img)
outimg = similar(img, size(img));
itp = interpolate(img, BSpline(Linear()))
for (k, opix) in enumerate(CartesianIndices(outimg))
if 1<=u[k]<=m && 1<= v[k]<= n
outimg[opix] = itp(u[k], v[k])
else
outimg[opix] = Gray(1.0)
end
end
save("outputimage.jpg", outimg)
2 Likes
I’d like to experiment with these features, but some of the following functions/files are missing in my environment.
what do I need to add to make these work…?
include("complextransf.jl");
...
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.15, B=1.15, C=-1.15, D=1.15));
invtform = get_invtform(obj, invf);
ComplexImageTransformations.jl is a small project I started during the pandemic, when I tried to understand how ImageTransformations work. It’s not online.
Mainly, the complex image transformation is defined as follows:
A mxn-image overlays a grid of the same resolution, on a rectangle [a,b]\times [c,d], from the complex plane.
A complex function, f, possibly multi-valued (like sqrt, log, asin, etc, but with a uni-valued inverse), transforms the pixels in the input image in the same way it transforms the corresponding grid points from the complex plane.
The output image, in turn, is superimposed on a mxn - grid of a minimal rectangle [A, B]\times [C,D], containing
f([a,b]\times[c,d]).
The inverse transformation, invtform, is defined by f^{-1}, composed at left and at right with affine transformations that define the correspondence pixel-> grid point, and conversely, for output, respectively input image.
Image transformation illustrates much better the properties of complex functions, especially multivalued ones, than presenting the effect of the same function on a grid of vertical and horizontal lines, drawn on a given rectangle.
Here is the file complextransf.jl https://gist.github.com/empet/b2447da03d99358fbf90cf2b039a6672.
Update: including complextransf.jl, the derivation of the vectors u, v is not necessary. u,v have been required by OP, to avoid using transformations and SVector(s).
Instead of using u, v, just call the function complexwarped:
include("complextransf.jl");
imgc = load("ai-boy.jpg")
img = Gray.(imgc)
invf(z::Complex) = z^2
obj = ImageRectangle(img, (a=-1, b=1, c=-1, d=1), (A=-1.1, B=1.1, C=-1.1, D=1.1));
imgw = complexwarped(obj, invf)
save("sqrt-image.jpg", imgw)
2 Likes