I’m very new to Makie, so there is probably something I did not understand.
I’m trying to build some sort of UI with Makie, in which the number of widgets will change over time. In order to handle this I tried an approach where I empty the Makie figure and recreate new widgets. However, this quickly becomes very inefficient (it gives me the impression that zombie widgets still exist from earlier versions of the scene).
A MWE illustrating this is the following “countdown timer”: even though there are less and less widgets, it takes more and more time to display them
julia> using GLMakie
julia> f = Figure()
julia> for i in 15:-1:1
@time begin
empty!(f)
for j in 1:i
Label(f[j,1], "$j")
end
end
end
2.370084 seconds (4.86 M allocations: 306.997 MiB, 3.96% gc time, 96.17% compilation time: 25% of which was recompilation)
0.102843 seconds (242.75 k allocations: 14.234 MiB, 48.58% compilation time: 33% of which was recompilation)
0.060081 seconds (174.01 k allocations: 9.734 MiB)
0.091768 seconds (158.91 k allocations: 8.907 MiB, 15.85% gc time)
0.116466 seconds (145.34 k allocations: 8.150 MiB)
0.178382 seconds (134.00 k allocations: 7.480 MiB)
0.291626 seconds (126.24 k allocations: 6.964 MiB)
0.520312 seconds (124.74 k allocations: 6.718 MiB)
0.903502 seconds (132.60 k allocations: 6.730 MiB, 1.59% gc time)
1.468623 seconds (155.38 k allocations: 7.494 MiB)
2.448408 seconds (200.61 k allocations: 8.872 MiB)
3.882972 seconds (276.48 k allocations: 11.876 MiB)
5.793398 seconds (385.54 k allocations: 15.622 MiB, 0.12% gc time)
7.856370 seconds (505.28 k allocations: 21.093 MiB)
7.769431 seconds (535.33 k allocations: 21.545 MiB, 0.07% gc time)
Hm we do test a couple things about deletion to hopefully remove all ties. But it’s possible something’s missing. Maybe one could do a heap snapshot to see what’s building up
But again I’m not sure how to interpret the results. For example:
does the Makie.Label line indicate that there are 15 instances of Makie.Label in memory? If so, this is neither what I would have expected (only one label remaining in the end) nor what I feared (120 labels accumulated over the 15 iterations)
also, I’m have no idea whether it is normal for there to be 16 instances of a Makie.Scene
However, maybe the following is more interesting: I took 2 different heap snapshots and tried to compare them. The first snapshot is taken after the 3rd iteration (when everything seems to be compiled, but things are still fast)
In both, I filtered everything related to Makie. It looks like the Makie.Screen object (which overall always takes the most space, apparently because it contains almost everything else) has seen its size increase by a factor of 8.
Inside it, the screens objects take up 5 times more space in the end than at the beginning. But perhaps even more interesting is the renderlist field, which goes from almost nothing to nearly half the size of the Makie.Screen object (its size has been multiplied by 265).
In your opinion, could that be the origin of my issue?
Maybe? Simon would have to say but if the renderlist is not cleared correctly that would at least not allow those plot primitives to be GC’ed. Although it’s weird that it works for Simon and not you. What system are you on?
julia> using GLMakie
g
julia> function go()
f = Figure()
for i in 15:-1:1
@time begin
empty!(f)
for j in 1:i
Label(f[j,1], "$j")
end
end
end
end
go (generic function with 1 method)
julia> go()
0.903762 seconds (3.03 M allocations: 185.278 MiB, 4.81% gc time, 98.02% compilation time)
0.024625 seconds (137.36 k allocations: 6.724 MiB, 60.83% compilation time)
0.017196 seconds (105.29 k allocations: 4.781 MiB, 46.93% gc time)
0.007553 seconds (95.24 k allocations: 4.344 MiB)
0.006975 seconds (85.62 k allocations: 3.923 MiB)
0.006105 seconds (76.32 k allocations: 3.513 MiB)
0.005620 seconds (67.33 k allocations: 3.114 MiB)
0.004727 seconds (58.65 k allocations: 2.727 MiB)
0.004196 seconds (50.29 k allocations: 2.352 MiB)
0.003704 seconds (42.24 k allocations: 1.987 MiB)
0.002903 seconds (34.50 k allocations: 1.633 MiB)
0.002200 seconds (27.08 k allocations: 1.289 MiB)
0.001664 seconds (19.97 k allocations: 979.258 KiB)
0.001081 seconds (13.18 k allocations: 649.297 KiB)
0.000568 seconds (6.69 k allocations: 330.320 KiB)
I’m not sure if it is that simple, but you could try screen = display(fig) and then look for renderlist in the attributes of screen and maybe empty! it? That might free the plot objects
using GLMakie
function foo()
f = Figure()
for i in 15:-1:1
@time begin
empty!(f)
for j in 1:i
Label(f[j,1], "$j")
end
end
screen = display(f)
empty!(screen.renderlist)
end
end
foo()
Behaviorwise, this does not change anything significantly: it causes no error, and the slow down is still there. The heap profile (taken at the end) does change significantly, though: now the renderlist takes almost no space, as expected
At least not the only one… The tests for this deletion behavior in Makie actually test without display so I’m not surprised they didn’t catch this. We should investigate and add tests for each backend.
I actually also didn’t look very closely at the example and just pasted it into a begin ... end block, not realizing it relied on being run line by line to display…
So I can actually reproduce it as well.
I don’t think the renderlist is the problem, it’s just where all the render objects live, and as far as I can tell, it gets emptied completely (but, since you run the empty!before the next loop, there will be lots of renderobjects in there at the end of the loop).
I do see lots of Observables{Any}(true) in the profile, after empty! & gc, which I think is the problem…
Gotta figure out where those are coming from.
That’s how I’d profile it:
using GLMakie
Profile.take_heapsnapshot("before.heapsnapshot")
begin
f = Figure()
display(f)
for i in 15:-1:1
for j in 1:i
Label(f[j, 1], "$j")
end
empty!(f)
end
end
Profile.take_heapsnapshot("after.heapsnapshot")
