GLMakie Method error (Makie.image!)

Specific Domains Visualization
1

I am trying to duplicate the example on the VideoIO github page that uses Makie. My code is very similar to this example but using some cell phone video from a Blackberry Key2.

using VideoIO
using GLMakie

path = string(homedir(), "\\OneDrive\\DanielShare\\Muskogee\\motion video")
cd(path)
f = VideoIO.openvideo("MOV_20210608_1540366.mp4")

img = read(f)
scene = Makie.Scene(resolution = reverse(size(img)))
makieimg = Makie.image!(scene, img, show_axis = false, scale_plot = true)[end]
Makie.rotate!(scene, -1pi)
display(scene)

while !eof(f)
	read!(f, img)
	makieimg[1] = img
	sleep(1/f.framerate)
end

This gives me the error message:

julia> includet("reviewmovie.jl")
ERROR: MethodError: no method matching lastindex(::Image{Tuple{IntervalSets.ClosedInterval{Float32}, IntervalSets.ClosedInterval{Float32}, Matrix{ColorTypes.RGB{Float32}}}})
Closest candidates are:
  lastindex(::Any, ::Any) at abstractarray.jl:348
  lastindex(::Union{ArrayInterface.BidiagonalIndex, ArrayInterface.TridiagonalIndex, ArrayInterface.BandedBlockBandedMatrixIndex, ArrayInterface.BandedMatrixIndex, ArrayInterface.BlockBandedMatrixIndex}) at C:\Users\jakez\.julia\packages\ArrayInterface\CYf5x\src\array_index.jl:203
  lastindex(::Union{Tables.AbstractColumns, Tables.AbstractRow}) at C:\Users\jakez\.julia\packages\Tables\YzCZp\src\Tables.jl:173
  ...
Stacktrace:
 [1] top-level scope
   @ C:\Users\jakez\OneDrive\DanielShare\Muskogee\motion video\reviewmovie.jl:10
in expression starting at C:\Users\jakez\OneDrive\DanielShare\Muskogee\motion video\reviewmovie.jl:10

julia>

Your ideas are appreciated.

2

I tried again with an exact duplicate of the sample program from VideoIO and get the same error.

The program used is below with the command Makie.image! causing the problem.

import Makie
import VideoIO

#io = VideoIO.open(video_file)
io = VideoIO.testvideo("annie_oakley") # for testing purposes
f = VideoIO.openvideo(io)

img = read(f)
scene = Makie.Scene(resolution = reverse(size(img)))
makieimg = Makie.image!(scene, img, show_axis = false, scale_plot = true)[end]
Makie.rotate!(scene, -0.5pi)
display(scene)

while !eof(f)
    read!(f, img)
    makieimg[1] = img
    sleep(1/f.framerate)
end
3

The example is outdated :wink: I’m not at a pc, but it should be easy to update the syntax with the docs!

4

I have to confess that this is my first use of Makie and VideoIO. I am looking through the docs, but nothing obvious yet.

5

Just experimenting here, if I create a tuple I can index into it as follows:

julia> a = (1, "a", 3.4)
(1, "a", 3.4)

julia> a[1]
1

julia> a[2]
"a"

julia> a[3]
3.4

julia> a[end]
3.4

julia> a[begin]
1

Then if I run a portion of the example code

import Makie
import VideoIO

#io = VideoIO.open(video_file)
io = VideoIO.testvideo("annie_oakley") # for testing purposes
f = VideoIO.openvideo(io)

img = read(f)
scene = Makie.Scene(resolution = reverse(size(img)))
makieimg = Makie.image!(scene, img, show_axis = false, scale_plot = true)

Note the [end] has been removed from the last line, this portion of the code runs. I can index into makieimg with digits within the square brackets but not with makieimg[end] or makieimg[begin]. If I run the remaining lines before the while loop nothing displays on my laptop but an image displays on another laptop. Mine is running Julia 1.6.1 and the other is 1.6.0, otherwise they are the same. The while loop does not run on either machine.

6

please take a look at the current documentation of Makie.
http://makie.juliaplots.org/stable/index.html

Scene is not longer the proper way of doing it. Probably, something like this will help

fig = Figure()
makieImg = image!(fig[1,1], img)
fig
7

I now have code that runs but eats up CPU cycles and memory. The code I now have is

using GLMakie
using VideoIO

io = VideoIO.testvideo("annie_oakley") # for testing purposes
f = VideoIO.openvideo(io)

img = read(f)
fig = Figure()
ax = Axis(fig[1,1], aspect = DataAspect())
hidedecorations!(ax)

makieimg = image!(rotr90(img))

framerate = 30
i = 0
while !eof(f)
    i += 1
    print("Frame $i\r")
    read!(f, img)
    makieimg = image!(rotr90(img))
    sleep(1/framerate)
end

when I look at the memory used after I run the program I obtain:

julia> varinfo()
  name                    size summary

  –––––––––––––––– ––––––––––– –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
  Base                         Module

  Core                         Module

  InteractiveUtils 272.678 KiB Module

  Main                         Module

  ans                  0 bytes typeof(varinfo)

  ax               801.817 MiB Axis

  f                  1.440 KiB VideoIO.VideoReader{true, VideoIO.SwsTransform, String}
  fig              801.817 MiB Figure
  framerate            8 bytes Int64
  i                    8 bytes Int64
  img              225.047 KiB 240×320 PermutedDimsArray(::Array{RGB{N0f8},2}, (2, 1)) with eltype ColorTypes.RGB{FixedPointNumbers.N0f8}
  io                 1.440 KiB VideoIO.AVInput{String}
  makieimg         801.817 MiB Image{Tuple{IntervalSets.ClosedInterval{Float32}, IntervalSets.ClosedInterval{Float32}, Matrix{ColorTypes.RGB{Float32}}}}
  subtypetree          0 bytes typeof(subtypetree)

julia>

Watching the task manager the memory consumption increases while the code is running. Watching the video as it progresses, the framerate decreases. When I play a cellphone video, the program totally stalls. I am sure there is a more efficient way of doing things, some help is appreciated.

8

You should use Nodes, e.g.

imgNode = Node(img)
then in your loop, you call something like

imgNode[] = read(f, img)

this should just update your figure… I didn’t tested but something like that should work.
http://makie.juliaplots.org/stable/animation.html#Animations

9

That was most helpful. I now have something that does not eat memory! The only thing currently missing is a foolproof way to get the framerate from the file. In the past it would have been f.framerate according to the demo code on the github VideoIO page but that seems not to work anymore. I have a workaround that works in some cases.
My current code is:

using GLMakie
using VideoIO
using Printf

#filename = "MOV_20210607_1540323.mp4"
#f = VideoIO.openvideo(filename)
io = VideoIO.testvideo("annie_oakley") # for testing purposes
f = VideoIO.openvideo(io)

framerate = 0.0
try
    framerate = VideoIO.get_number_frames(filename) / VideoIO.get_duration(filename)
    println("Using calculated framerate of $framerate")
catch
    framerate = 30.0
    println("Using default framerate of $framerate")
end
@show(framerate)
img = read(f)
fig = Figure()
ax = Axis(fig[1,1], aspect = DataAspect())
hidedecorations!(ax)
node = Node(rotr90(img))

makieimg = image!(node)

i = 0
while !eof(f)
    i += 1
    print("Frame $i\r")
    read!(f, img)
    node[] = rotr90(img)
    save(@sprintf("test_%04d.png", i) , fig.scene)
    sleep(1/framerate)
end

Thank you very much for your help!

Edit: minor correction and added a line to save every frame to a png file.

1 Like
10

One more minor correction, the first line after try should read

framerate = (VideoIO.get_number_frames(filename) - 1) / VideoIO.get_duration(filename)