I’m planning to add photometric optimization (think Lucas-Kanade style model to image alignment, direct methods for visual odometry & SLAM, etc.) to my VisualGeometryOptimization package. I’d like to not re-invent the wheel, the wheel here being basic image structures and utilities for creating image pyramids and interpolating samples with gradients.
Looking at what’s available, I can see Images.jl for the basic image container, and Interpolations.jl for sampling, but the latter doesn’t work on images from the former, which instead need to be converted to an array.
It seems that Images.jl doesn’t have its own way to get samples from an image (using a user-defined interpolation scheme), except when applying a transformation to a whole image. Is that right?
Also, Images.jl and JuliaImages don’t seem to have a way to create, store and access image pyramids. There is a Pyramids.jl package, but this hasn’t been updated in 7 years.
I wrote the PyramidScheme.jl package last year to handle and create pyramids. This is currently focused on geospatial data. I would be happy to extend this to more use cases.
The image should be an n-channel 2D array (n=1 or 3, but ideally any n is supported), supporting 8 and possibly 16 bit colour depths per channel. I will be performing bilinear sampling on the images, requiring both values (one per channel) and gradients (two per channel). I would also like to construct an image pyramid for each image, where each level of the pyramid is an x2 downsampled version of the (larger) image below.
My ultimate goal is model to image alignment using photometric optimization, for tracking objects in video, and tracking the world (i.e. visual odometry & SLAM) also.