Makie - Observables and removing widgets

Specific Domains Visualization
1

I am trying to build an interactive figure (GLMakie v0.6.13, Makie v0.17.13) where the specific layout, including the existence of various subplots, depends on the data. I would like to use Makie’s predefined widgets to filter my data and then rebuild the full figure.

I need some guidance on how to structure this type of application since I suspect that my current approach is not idiomatic w.r.t. Makie.

This is the part that works fine: I recursively build my figure by handing down the Figure through my method calls and then attaching any new axes and widgets to it, e.g. Textbox(figure). On the way up, I organize these objects into GridLayout()s and return them to be inserted by the caller, until I reach the top level, where I attach these layouts to the Figure, e.g. figure[1, 1] = build_subdisplay!(figure; data).

However, when I now try to interactively rebuild the figure in response to some Observable triggered by a Makie widget, weird stuff starts to happen. For example:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.3 (2022-05-06)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

julia> using GLMakie

julia> function rebuild!(f::Figure)
           empty!(f)
           textbox = Textbox(f)
           on(textbox.stored_string) do s
               rebuild!(f)
           end
           f[1, 1] = textbox
       end
rebuild! (generic function with 1 method)

julia> f = Figure()

julia> rebuild!(f)
Textbox()

If I now trigger stored_string with an empty textbox, I get an Error in callback: BoundsError: attempt to access 0-element Vector{AbstractPlot} at index [1]

Summary 
Stacktrace:
  [1] getindex
    @ ./array.jl:861 [inlined]
  [2] (::Makie.var"#1743#1760"{Label, Observable{Int64}})(ci::Int64, bbs::Vector{GeometryBasics.HyperRectangle{2, Float32}})
    @ Makie ~/.julia/packages/Makie/Ppzqh/src/makielayout/blocks/textbox.jl:74
  [3] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
  [4] invokelatest
    @ ./essentials.jl:714 [inlined]
  [5] #15
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:396 [inlined]
  [6] (::Observables.var"#callback#13"{Observables.var"#15#16"{Makie.var"#1743#1760"{Label, Observable{Int64}}, Observable{Vector{Point{2, Float32}}}}, Tuple{Observable{Int64}, Observable{Vector{GeometryBasics.HyperRectangle{2, Float32}}}}})(x::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:342
  [7] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
  [8] invokelatest
    @ ./essentials.jl:714 [inlined]
  [9] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [10] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86
 [11] #15
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:396 [inlined]
 [12] (::Observables.var"#callback#13"{Observables.var"#15#16"{Makie.var"#1742#1759", Observable{Vector{GeometryBasics.HyperRectangle{2, Float32}}}}, Tuple{Observable{Any}, Observable{Vector{Point{3, Float32}}}}})(x::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:342
 [13] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [14] invokelatest
    @ ./essentials.jl:714 [inlined]
 [15] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [16] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86
 [17] (::Observables.var"#8#9"{Observable{Any}})(x::String)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:123
 [18] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [19] invokelatest
    @ ./essentials.jl:714 [inlined]
 [20] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [21] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86
 [22] (::Observables.var"#8#9"{Observable{Any}})(x::String)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:123
 [23] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [24] invokelatest
    @ ./essentials.jl:714 [inlined]
 [25] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [26] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86
 [27] #15
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:396 [inlined]
 [28] (::Observables.var"#callback#13"{Observables.var"#15#16"{Makie.var"#1018#1019"{DataType}, Observable{Any}}, Tuple{Observable{Any}}})(x::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:342
 [29] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [30] invokelatest
    @ ./essentials.jl:714 [inlined]
 [31] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [32] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86
 [33] defocus!(tb::Textbox)
    @ Makie ~/.julia/packages/Makie/Ppzqh/src/makielayout/blocks/textbox.jl:346
 [34] (::Makie.var"#1752#1775"{Textbox, Makie.var"#cursor_backward#1774"{Observable{Int64}}, Makie.var"#cursor_forward#1773"{Textbox, Observable{Int64}}, Makie.var"#removechar!#1770"{Textbox, Observable{Vector{Char}}, Observable{Int64}}, Observable{Bool}, Observable{Int64}})(event::Makie.KeyEvent)
    @ Makie ~/.julia/packages/Makie/Ppzqh/src/makielayout/blocks/textbox.jl:225
 [35] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [36] invokelatest
    @ ./essentials.jl:714 [inlined]
 [37] notify
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:146 [inlined]
 [38] setindex!
    @ ~/.julia/packages/Observables/jbVpe/src/Observables.jl:86 [inlined]
 [39] (::GLMakie.var"#keyoardbuttons#149"{Observable{Makie.KeyEvent}})(window::GLFW.Window, button::GLFW.Key, scancode::Int32, action::GLFW.Action, mods::Int32)
    @ GLMakie ~/.julia/packages/GLMakie/K6iJk/src/events.jl:114
 [40] _KeyCallbackWrapper(window::GLFW.Window, key::GLFW.Key, scancode::Int32, action::GLFW.Action, mods::Int32)
    @ GLFW ~/.julia/packages/GLFW/BWxfF/src/callback.jl:43
 [41] PollEvents
    @ ~/.julia/packages/GLFW/BWxfF/src/glfw3.jl:620 [inlined]
 [42] pollevents(screen::GLMakie.Screen)
    @ GLMakie ~/.julia/packages/GLMakie/K6iJk/src/screen.jl:50
 [43] fps_renderloop(screen::GLMakie.Screen, framerate::Float64)
    @ GLMakie ~/.julia/packages/GLMakie/K6iJk/src/rendering.jl:20
 [44] renderloop(screen::GLMakie.Screen; framerate::Float64)
    @ GLMakie ~/.julia/packages/GLMakie/K6iJk/src/rendering.jl:46
 [45] renderloop(screen::GLMakie.Screen)
    @ GLMakie ~/.julia/packages/GLMakie/K6iJk/src/rendering.jl:39
 [46] (::GLMakie.var"#126#128"{GLMakie.Screen})()
    @ GLMakie ./task.jl:429

(The error gets raised from here, which looks like something goes wrong with the label if the textbox contains no text. However, I cannot reproduce this problem without my rebuild-from-observable setup.)

In my more complex figure, other stuff starts to go wrong, e.g. click-and-draw to zoom stops working properly (some mouse events seem to go missing).

I suspect that Base.empty!(::Figure) does not completely remove the old widgets’ callbacks and that these still get triggered from somewhere in the render loop. But more generally, I guess that I am just using Makie incorrectly.

How should I build an application like this? (I would like to rebuild the full scene after some user interactions because its structure / composition may have changed completely.)

2

In case anyone else runs into this, it seems to be related to widgets not being disconnected properly, in this case specifically the Textboxes. See for example this comment on GitHub.

I ended up just completely replacing the Figure and redisplaying it, which works fine. In my case, since I was waiting for the window to close in the main script, this required a little additional work to relink the callback, e.g.:

closed = Base.Event()
selection = Observable(...)  # this triggers and controls figure (re)construction

window_open_callback = Ref{Any}(nothing)
on(selection) do selected
    # filter data etc...
    figure = build_figure(data; selection)

    isnothing(window_open_callback[]) || off(window_open_callback[])
    display(figure)
    window_open_callback[] = on(events(figure).window_open) do is_open
        !is_open && notify(closed)
    end
end

notify(selection)
wait(closed)
3

This broke in Makie v0.18, but @sdanisch showed me a better solution:

screen = GLMakie.Screen()
selection = Observable(...)  # this triggers and controls figure (re)construction

on(selection) do selected
    # filter data etc...
    figure = build_figure(data; selection)
    display(screen, figure)
end

notify(selection)
wait(screen)