I’m trying to speed-up the following piece of code for an art project.
I have a collection of png images. All of them together is 25GB+
And in a loop, I need to keep copying an arbitrary pixel in one of the png images (simplified code below), right now it takes running overnight (~9 hours) to complete even with some preloading of images as much as I can.
How can I do this much faster possibly by preprocessing the images into a faster to access format, or possibly accessing the desired pixel without loading the entire image?
EDIT: Right now I’m also thinking about preprocessing it all into HDF5 and maybe that speeds it up
Here’s the basic code:
for frame = 1:Ntotalframes newimg = copy(preloaded_img_of_proper_dims) for j = 1:ysize for i = 1:xsize # Access and copy a single pixel in one of the png images in the 25GB collection file_get_to_from, pixel_coords_in_file = getDesiredFilenameAndPixelIndex(i, j, frame) newimg[i, j] = getpixelvalue(file_get_to_from, pixel_coords_in_file) # Basically I'm doing the code below (but with some preloading). How should I speed it up? # I can't preload preload entire 25 GB+ images into RAM # getpixelvalue = load(file_get_to_from)[pixel_coords_in_file, pixel_coords_in_file] end end newimg_file_name = @sprintf("%04d.png", frame); save(newimg_file_name, newimg) end