Creating a video from a stack of Images

Specific Domains Signal and Image Processing
1

VideoIO.jl only seems to have the capabilities of reading in video with ffmpeg.
I couldn’t find any options to create a Video so I was wondering if I am missing something.

I know @sdanisch is exporting to png files first and then uses ffmpeg to create a video. That is not really an option for me since my image data is quite large.

2

ffmpeg certainly has the capability to produce videos, and I had intended on adding it to VideoIO.jl a long time ago, but it’s been challenging for me to make the time for that package recently. Pull requests are very welcome.

In the mean time, you might check to see if one of the OpenCV.jl implementations implements video. I know that OpenCV supports it, but I don’t know if that functionality has been exposed in either package.

(I’m also not sure how easy it is to install either package, as neither is registered.)

Sorry not to be more helpful.

Cheers,
Kevin

1 Like
3

Hi, I now use Images.jl or actually ImageMagick.jl for this task which works pretty well. You just have to convert the data first to an RGB Image and then simply call save. But I think this just works for animated gifs, which are in my case pretty suitable.

4

GR.jl can produce videos on the fly. You can find some examples in this talk (e.g. slides 10, 11). There’s also a backend for animated GIFs - but creating videos (with only 256 colors) is probably not what you want …

1 Like
5

I looked around about a year ago, and the best I could do was to use matplotlib.animation.FuncAnimation via PyCall. I wouldn’t recommend it, but I can post my code if the other options don’t work for you.

6

Thank you all for the suggestions I solved my particular problem the following way

open(`ffmpeg -f rawvideo -pix_fmt gray -s:v 5120x5120 -r 70 -i pipe:0 test.mkv`, "w") do out
    for frame in video
        img = convert(Image{Gray{U8}}, frame)
        write(out, reinterpret(UInt8, data(img)))
    end
end

It is not terrible performant but it get’s the job done for now.

7 Likes
7

This is great! I updated and generalized it a bit:

using Images

function writevideo(imgstack, filename; overwrite=true, framerate=30)
    # Input validation
    length(size(imgstack)) == 3 || error("input image must have three dimensions")
    if !(eltype(imgstack) <: Color)
        error("element type of input array is $(eltype(imgstack));
         needs to be RGB or Gray")
     end

    # Translate inputs to command line args
    ow = overwrite ? "-y" : ""
    w, h, nframes = size(imgstack)

    if eltype(imgstack) <: RGB
        open(`ffmpeg $ow -f rawvideo -pix_fmt rgb24 -s:v $(w)x$(h)
            -r $framerate -i pipe:0 -vf "transpose=0" $filename`, "w") do out
            for i = 1:nframes
                write(out, convert.(RGB24, imgstack[:,:,i]))
            end
        end
    else
        open(`ffmpeg $ow -f rawvideo -pix_fmt gray -s:v $(w)x$(h)
            -r $framerate -i pipe:0 -vf "transpose=0" $filename`, "w") do out
            for i = 1:nframes
                write(out, convert.(Gray{N0f8}, imgstack[:,:,i]))
            end
        end
    end
end
2 Likes
8

Further modified (after much head-bashing) to allow for near-universally-compatible output:

using Images

function writevideo(fname, imgstack::Array{<:Color,3};
                    overwrite=true, fps=30::UInt, options=``)
    ow = overwrite ? `-y` : `-n`
    h, w, nframes = size(imgstack)

    open(`ffmpeg
            -loglevel warning
            $ow
            -f rawvideo
            -pix_fmt rgb24
            -s:v $(h)x$(w)
            -r $fps
            -i pipe:0
            $options
            -vf "transpose=0"
            -pix_fmt yuv420p
            $fname`, "w") do out
        for i = 1:nframes
            write(out, convert.(RGB{N0f8}, clamp01.(imgstack[:,:,i])))
        end
    end
end

@kevin.squire: worth making a PR to VideoIO?

10 Likes
9

Hi All, this function is great (thanks!) – if there are no objections, I added the latest writevideo function (with minor changes/fixes) to a registered package here:
https://github.com/JuliaRobotics/Caesar.jl/blob/fce7ce982b96c40d39aa7771283ffbf132a533c5/src/images/images.jl#L13

Also a shout out to Video4Linux.jl which might be of use to folks:
https://github.com/Affie/Video4Linux.jl

1 Like
10

Have you tried VideoIO.jl? If that works for your use case, it’ll be more robust & user-friendly to add that as a lazy-loaded dependency than requiring people to install FFMPEG themselves.

2 Likes
11

There’s also FFMPEG.jl if you didn’t want to switch to the convenience functions VideoIO provides.

12

I haven’t tried VideoIO.jl recently but looks to be a good solution for writing videos thanks!