What packages for images, image pyramids and interpolation do you recommend?

I’m planning to add photometric optimization (think Lucas-Kanade style model to image alignment, direct methods for visual odometry & SLAM, etc.) to my VisualGeometryOptimization package. I’d like to not re-invent the wheel, the wheel here being basic image structures and utilities for creating image pyramids and interpolating samples with gradients.

Looking at what’s available, I can see Images.jl for the basic image container, and Interpolations.jl for sampling, but the latter doesn’t work on images from the former, which instead need to be converted to an array.

It seems that Images.jl doesn’t have its own way to get samples from an image (using a user-defined interpolation scheme), except when applying a transformation to a whole image. Is that right?

Also, Images.jl and JuliaImages don’t seem to have a way to create, store and access image pyramids. There is a Pyramids.jl package, but this hasn’t been updated in 7 years.

Does anyone have any advice on what to use?

1 Like

I wrote the PyramidScheme.jl package last year to handle and create pyramids. This is currently focused on geospatial data. I would be happy to extend this to more use cases.

1 Like

@user664303 what are the requirements for the representation of the image and for the operations you have in mind?

The image should be an n-channel 2D array (n=1 or 3, but ideally any n is supported), supporting 8 and possibly 16 bit colour depths per channel. I will be performing bilinear sampling on the images, requiring both values (one per channel) and gradients (two per channel). I would also like to construct an image pyramid for each image, where each level of the pyramid is an x2 downsampled version of the (larger) image below.

You mean bilinear interpolation?

How do you plan to use the data structure? What is your ultimate goal?

Bilinear interpolation, yes.

My ultimate goal is model to image alignment using photometric optimization, for tracking objects in video, and tracking the world (i.e. visual odometry & SLAM) also.