[GLMakie] Overlay plots in animated 3D scene?

I am now back at an old problem which I abandoned due to lack of time: displaying overlay plots (2D) in an animated 3D scene :wink:

I started a GH discussion back then and I got it almost working with the help of Simon (still could not fix the layering issue, the plot was always behind the 3D scene) but things have apparently changed now and I cannot even reproduce the old layout.

Just a quick reminder, this is what I am looking for (roughly):

The proposed code last year was

    subwindow = Scene(scene, px_area=Observable(Rect(100, 100, 200, 200)), clear=true, backgroundcolor=:green)
    subwindow.clear = true
    meshscatter!(subwindow, rand(Point3f, 10), color=:gray)
    plot!(subwindow, [1, 2, 3], rand(3))

which kind of worked and produced in the old application an overlay like this:

When I however try the same with the current version of GLMakie (v0.10.5) I get a full overlay of the plot and it’s still somehow behind the 3D scene, or at least some parts, it’s a bit weird:

Two questions:

  1. How do I adjust the size and position of the overlay plot correctly?
  2. How do I put the plot in the foreground so that the 3D scene is entirely behind the plot?

Any hints are highly appreciated, I tried so many things already :see_no_evil:

Here is a MWE which can be executed by running:

julia> include("scripts/uioverlayplot.jl")
eventloop (generic function with 1 method)

julia> app = initialize()

It’s a rough skeleton of the main application which the mentioned code block (four lines) which adds the subwindow:

using Makie
using GLMakie
using GLFW
using GeometryBasics


@kwdef mutable struct App
    scene::Scene = Scene(backgroundcolor=RGBf(0.9))
    cam::Makie.Camera3D = cam3d!(scene, rotation_center = :lookat)
    fps::Int = 60
end

"""
Draws a grid on the XY-plane with an optional `center` point, `span`, grid-`spacing` and
styling options.
"""
function basegrid!(app::App; center=(0, 0, 0), span=(-10, 10), spacing=1, linewidth=1, color=(:grey, 0.3))
    scene = app.scene
    min, max = span
    center = Point3d(center)
    for q ∈ range(min, max; step=spacing)
        lines!(scene, [Point3d(q, min, 0) + center, Point3d(q, max, 0) + center], color=color, linewidth=linewidth)
        lines!(scene, [Point3d(min, q, 0) + center, Point3d(max, q, 0) + center], color=color, linewidth=linewidth)
    end
    app
end



"""

Initialise the app, enter the event loop and return the app instance to
the REPL.

"""
function initialize()
    app = App()
    #center!(app.scene)
    update_cam!(app.scene, app.cam, Vec3f(5), Vec3f(0, 0, 0), Vec3f(0, 0, 1))

    n = 10

    meshscatter!(
        app.scene,
        rand(Vec3f, n),
        markersize = [rand()/10 for _ in 1:n]
    )

    lines!(app.scene, [rand(Vec3f) for _ in 1:n]; color=:red, linewidth=4)

    basegrid!(app)

    Threads.@spawn :interactive eventloop(app)
    app
end


function eventloop(app::App)
    screen = display(GLMakie.Screen(start_renderloop=false, focus_on_show=true, title="RainbowAlga"), app.scene)
    glw = screen.glscreen
    GLMakie.GLFW.SetWindowPos(glw, 100, 100)
    GLMakie.GLFW.SetWindowSize(glw, 400, 400)

    scene = app.scene

    # Adding the overlay plot
    subwindow = Scene(scene, px_area=Observable(Rect(100, 100, 200, 200)), clear=true, backgroundcolor=:green)
    subwindow.clear = true
    meshscatter!(subwindow, rand(Point3f, 10), color=:gray)
    plot!(subwindow, [1, 2, 3], rand(3))

    while isopen(screen)
        frame_start = time()
        rotate_cam!(scene, Vec3f(0, 0.005, 0))

        GLMakie.pollevents(screen)
        GLMakie.render_frame(screen)
        GLMakie.GLFW.SwapBuffers(GLMakie.to_native(screen))

        yield()

        sleep_time = 1.0/app.fps - (time() - frame_start)
        sleep_time > 0 && sleep(sleep_time)
    end

    GLMakie.destroy!(screen)
end
1 Like

px_area is called viewport now, does that help?

1 Like

Yes, now the position is correct again :smiley:

The missing piece is how to put things in foreground…

I mean, I would assume that (z?) layering is following the order of things added to the scene and the 2D overlay is the last I add.
Maybe the GLMakie.GLFW.SwapBuffers is causing the issue in the eventloop (isopen(screen)) while loop?

You could try setting depth_shift=1 (or 0, not sure which end of the scale) for the Scene of the overlay, or the plots within that Scene. Alternatively you can try simply translate!(subwindow, 0, 0, 9_999) or something which should get the subscene to be as close to the camera as is allowed…

depth_shift = 0.0 — adjusts the depth value of a plot after all other transformations, i.e. in clip space, where 0 <= depth <= 1. This only applies to
  GLMakie and WGLMakie and can be used to adjust render order (like a tunable overdraw).
1 Like

None of them work, unfortunately, I tried different settings and orders…

If I use translate!(subwindow, 0, 0, 9_999), the dots in the plot disappear, but I can still see the scene behind it :confused: