BayeredImages.jl (currently unregistered)
This announcement is somewhat premature, but I realized there’s a gap in the Julia image ecosystem that I’m surprised has not been filled already.
The problem
Color cameras use color filter arrays, like the Bayer matrix, or if you’re weird like Fujifilm, the X-Trans matrix. The raw image is really black and white; demosaicing algorithms convert this filtered image to a color image with RGB values at every pixel.
Depending on what kind of image processing you want to do, you might prefer to manipulate the raw image data before demosaicing. A great example of a kind of manipulation which would be useful to do before demosaicing is filtering hot pixels. If they’re not removed before the debayering step, the hot pixel’s data will spill into neighboring pixels, leading to artifacts that are annoying to clean up.
A median filter can handle this (especially if you have a map of hot pixels from the sensor), but applying it to raw CFA data needs care, since you probably only want to take the median with respect to pixels of the same color channel. Or perhaps you want to incorporate data from other channels (since the filter bandpasses overlap), but you’ll probably want to use something other than a simple median for this.
Goals
The goal is to provide a package that
- has types that represent color filter arrays, including efficient representations for the most common CFAs (like the Bayer matrix), and generic representations for wackier/experimental CFAs.
- has a type to represent mosaiced images, and correctly handles image transforms which may alter how the CFA is associated with the data (rotation, reflection, cropping).
- provides demosaicing algorithms, including generic ones that work with arbitrary CFAs (but are superior to ordinary interpolation methods).
Additionally, I’d like to provide versions of common image operations (such as the median filter and the Drizzle algorithm) adapted to work with raw color data, but complex algorithms might be better off placed in a separate package or as a package extension for existing image processing tools. I’ll probably include a few simple examples in this package though; I suspect bad pixel mitigation with median filtering would be handy for anyone interested in this package.
Calling all photographers!
It turns out I actually don’t use a color camera for the vast majority of my imaging. I’m an astrophotographer, and I use a monochome camera. I do have access to some raw image data from cameras I’ve borrowed from my university library, but it’s almost entirely astrophotographic data (time lapses, star trails, etc). I’m writing this package as part of an effort to eventually perform all of my astrophotographic processing in Julia, but I want this package to be more broadly usable for anyone using a color camera.
So if you have raw image data you’d be willing to share for internal testing, I’d be happy to use it! (Sadly, Wikimedia Commons does not host raw image data.) Image I/O is not within the scope of this package, but I’m interested in performing tests with raw data formats I don’t currently have access to (anything that’s not an Adobe DNG or Canon *.CR2 and *.CR3, and especially any color camera not using a standard Bayer matrix, like Fujifilm X-Trans cameras), and with non-astronomical subjects.