How to display moving objects in Makie.jl

ufechner7 · January 30, 2021, 2:40pm

Hello,

I am now able to display a 3D scene in the way I want it. But how can I display a moving system?

My main loop looks like this:

    for i = 0:4
        # calculate a vector of 3D coordinates
        X = range(0, stop=10, length=SEGMENTS+1)
        Y = zeros(length(X)) 
        Z = (10 .* cosh.(X./10) .- 10) * i/5.0 

        # loop over the particles of the main tether and render them as spheres
        for i in range(1, length=length(X))
            mesh!(scene3D, Sphere(Point3f0(X[i], Y[i], Z[i]), 0.07 * SCALE), color=:yellow)
        end

        sleep(0.2)
    end

This approach has one problem:

when displaying the scene after a change of the positions of the objects the old position is not deleted, you kind of see the trace of the objects, which I do not want

Any ideas how to fix this?

Full code available at: GitHub - ufechner7/KiteViewer: 3D viewer and simulator for airborne wind energy systems

To reproduce my problem, execute:

include("src/Minimal.jl")
main()

ufechner7 · January 30, 2021, 3:53pm

Further simplified, self-contained example:

using GeometryBasics, GLMakie

const SCALE = 1.2
const SEGMENTS = 7                    # number of tether segments

function create_coordinate_system(scene, points = 10, length = 10)
    # create origin
    mesh!(scene, Sphere(Point3f0(0, 0, 0), 0.1 * SCALE), color=RGBf0(0.7, 0.7, 0.7))
      
    # create y-axis in green
    points -= 3
    for y in range(0, length = 2points + 1)
        if y - points != 0
            mesh!(scene, Sphere(Point3f0(0, (y - points) * SCALE, 0), 0.1 * SCALE), color=:green)
        end
    end
    mesh!(scene, Cylinder(Point3f0(0, -(points+1) * SCALE, 0), Point3f0(0, (points+1) * SCALE , 0), Float32(0.05 * SCALE)), color=:green)
    for i in range(0, length=10)
        start = Point3f0(0, (points+1 + 0.07 * (i-0.5)) * SCALE, 0)
        stop = Point3f0(0, (points+1 + 0.07 * (i+0.5)) * SCALE, 0)
        mesh!(scene, Cylinder(start, stop, Float32(0.018 * (10 - i) * SCALE)), color=:green)
    end
end

function main()
    scene, layout = layoutscene(resolution = (840, 900), backgroundcolor = RGBf0(0.7, 0.8, 1))
    scene3D = LScene(scene, scenekw = (show_axis=false, limits = Rect(-7,-10.0,0, 11,10,11), resolution = (800, 800)), raw=false)
    create_coordinate_system(scene3D)

    cam = cameracontrols(scene3D.scene)
    cam.lookat[] = [0,0,5]
    cam.eyeposition[] = [-15,-15,5]
    update_cam!(scene3D.scene)

    layout[1, 1] = scene3D
    layout[2, 1] = buttongrid = GridLayout(tellwidth = false)
    buttongrid[1, 1:1] = [Button(scene, label = "RESET")]

    display(scene)

    for i = 0:4
        # calculate a vector of 3D coordinates
        X = range(0, stop=10, length=SEGMENTS+1)
        Y = zeros(length(X)) 
        Z = (10 .* cosh.(X./10) .- 10) * i/5.0 

        # loop over the particles of the main tether and render them as spheres
        for i in range(1, length=length(X))
            mesh!(scene3D, Sphere(Point3f0(X[i], Y[i], Z[i]), 0.07 * SCALE), color=:yellow)
        end

        sleep(0.2)
    end
    
    return nothing

I fixed two of my three problems already myself, no more flickering, no need to update the camera each time step.

But how can I make sure the old tether is deleted when I display the new tether?

Alternatively, how can I create the tether once and just move the coordinates of the spheres?

ufechner7 · January 30, 2021, 6:34pm

I found a solution myself. This works:

function main()
    scene, layout = layoutscene(resolution = (840, 900), backgroundcolor = RGBf0(0.7, 0.8, 1))
    scene3D = LScene(scene, scenekw = (show_axis=false, limits = Rect(-7,-10.0,0, 11,10,11), resolution = (800, 800)), raw=false)
    create_coordinate_system(scene3D)

    cam = cameracontrols(scene3D.scene)
    cam.lookat[] = [0,0,5]
    cam.eyeposition[] = [-15,-15,5]
    update_cam!(scene3D.scene)

    layout[1, 1] = scene3D
    layout[2, 1] = buttongrid = GridLayout(tellwidth = false)
    buttongrid[1, 1:1] = [Button(scene, label = "RESET")]

    display(scene)

    particles = Vector{AbstractPlotting.Mesh}(undef, SEGMENTS+1)

    for i = 0:4
        # calculate a vector of 3D coordinates
        X = range(0, stop=10, length=SEGMENTS+1)
        Y = zeros(length(X)) 
        Z = (10 .* cosh.(X./10) .- 10) * i/4.0 

        # loop over the particles of the main tether and render them as spheres
        if i == 0
            for j in range(1, length=length(X))
                particle = mesh!(scene3D, Sphere(Point3f0(X[j], Y[j], Z[j]), 0.07 * SCALE), color=:yellow)
                particles[j] = particle
            end
        else
            j=1
            for particle in particles
                translate!(particle, X[j], Y[j], Z[j])
                j += 1
            end
        end
        sleep(0.2)
    end
    
    return nothing
end

Probably it could still be simplified.

sijo · January 30, 2021, 7:16pm

Any reason you’re not using Nodes? (as described in https://makie.juliaplots.org/stable/animation.html and https://makie.juliaplots.org/stable/interaction.html )

ufechner7 · January 30, 2021, 8:29pm

As mentioned before, I am porting code from cgkit (a Python library) to Makie. Furthermore I do not know how to attach a Node to a 3D sphere. Finally my goal is to visualize incoming simulation data in realtime and I don’t know if Nodes can help with that.

But I would be very glad if someone could give me an example how to attach a 3D sphere to a Node.

ufechner7 · January 30, 2021, 10:11pm

I found out that moving and turning objects is not sufficient as soon as I want to connect the spheres with cylinders, because the length of the cylinders has to change when updating the scene, and that is not possible with the operations “rotate” and “translate”.

Even worse: The translate! function is not working correctly in 3D. I will create a bug report tomorrow.

So how can I either change the length of a cylinder, or remove it from the scene to recreate it?

ufechner7 · January 30, 2021, 11:29pm

I found the solution myself: The following command can be used to delete a mesh, for example a segment (=cylinder) from the scene:

delete!(scene, segment)

ffreyer · January 31, 2021, 12:11am

Deleting and re-adding meshes will have a lot of overhead.

If you’re looking to plot multiple instances of the same mesh you should use meshscatter(positions, marker=my_mesh) with positions = Node(Point3f0[[x1, y1, z1], [x2, y2, z2], ...]). You can update the positions via positions[] = new_positions is a new array of Point3f0s. You can also pass an array of Quarternions as rotation and scales via markersize. Both can be adjusted the same way.

If you’re looking to animate various unique meshes or meshes with unique textures you need to rely on mesh(my_mesh). I’m not sure if that has a position attribute off the top of my head. If it doesn’t or if you want to do more complex things you can pass a model matrix, again as a Node. There are some unexported functions in AbstractPlotting that should help with this, like translationmatrix, scalematrix and rotationmatrix.

In the end you probably want something like this:

using GLMakie, GeometryBasics
fig = Figure()
scene = fig[1, 1] = LScene(fig, scenekw =(camera=cam3d!,))

positions = Node([Point3f0(x, 0, 0) for x in 1:3:10])
meshscatter!(scene, positions, marker=Sphere(Point3f0(0), 0.3f0), markersize=1f0, color=[:red, :green, :blue])
meshscatter!(scene, positions, marker=Cylinder(Point3f0(0, -1, 0), Point3f0(0, 0.0, 0), 0.2f0), color=[:red, :green, :blue], markersize=1f0)
display(fig)

for t in range(0.0, 10*2pi, length=300)
    positions[] = [Point3f0(x+cos(t), 0, sin(t)) for x in 1:3:10]
    sleep(1/60)
end

ufechner7 · January 31, 2021, 12:55am

Thanks a lot for your example code!

But it will not solve my problem. One problem is that the length of the cylinders that I need is varying, depending on the ball distances. And there is no cylinder object in Makie that has length and diameter as parameters.

The following code works, and is good enough for me for now, but as you said it has a lot of overhead:

# globals
const SCALE = 1.2 
const KITE = FileIO.load("data/kite.obj")
const PARTICLES = Vector{AbstractPlotting.Mesh}(undef, SEGMENTS+1)
const SEGS      = Vector{AbstractPlotting.Mesh}(undef, SEGMENTS)
const KITE_MESH = Vector{MeshScatter{Tuple{Vector{Point{3, Float32}}}}}(undef, 1)
const init      = [false]

# draw the kite power system, consisting of the tether and the kite
function draw_system(scene, state)
    # loop over the particles of the main tether and render them as spheres
    for i in range(1, length=length(state.X))
        if init[1] 
            delete!(scene.scene, PARTICLES[i])
        end
        particle = mesh!(scene, Sphere(Point3f0(state.X[i], state.Y[i], state.Z[i]), 0.07 * SCALE), color=:yellow)
        PARTICLES[i] = particle
    end

    end_point = Point3f0(0,0,0)
    # loop over the springs of the main tether and render them as cylinders
    for i in range(1, length=length(state.X) - 1)
        if init[1] 
            delete!(scene.scene, SEGS[i])
        end
        start_point = Point3f0(state.X[i], state.Y[i], state.Z[i])
        end_point  = Point3f0(state.X[i+1], state.Y[i+1], state.Z[i+1])
        segment = mesh!(scene, Cylinder(start_point, end_point, Float32(0.035 * SCALE)), color=:yellow)
        SEGS[i] = segment
    end

    # rot = Quaternionf0(1, 0, -1, 0)
    # # kite.rot = rot3d(vec3(0, -1, 0), vec3(1, 0, 0), vec3(0, 0, -1), x, y, z)
    # # render the kite
    if init[1]
        delete!(scene.scene,  KITE_MESH[1])
    end
    KITE_MESH[1] = meshscatter!(scene, end_point, marker=KITE, markersize = 0.5, rotations = Vec3f0.(0, -1, 0), color=:blue)
    init[1] = true
end

ffreyer · January 31, 2021, 1:46am

Ok, for this you can use markersize. Assuming you also want to rotate the cylinders it’s probably best to have something like this:

m = Cylinder(Point3f0(0), Point3f0(0, 0, 1), 0.1)
pos = Node(Point3f0[...])
rots = Node(Vec3f0[...])
ms = Node(Vec3f0[...])
meshscatter!(pos, marker=m, rotations=rots, markersize=ms)

Vec3f0 rotations will rotate something pointing in z direction to where ever that vector points. So this would just be normalize. (end_point .- start_point).

Vec3f0 markersizes apply like a .*, so Vec3f0(1, 1, norm.(end_points .- start_points)) should do what you want, I think.

tamasgal · June 25, 2024, 6:03pm

Sorry for reviving this but Node is not defined anymore. What’s the current way of changing positions of objects inside a mesh?

I use already a_meshscatter.color = [...] and and a_meshscatter.markersize = [...] and both work fine (they propagate to the observables(?) but changing the position would require access to those observables and I cannot find them in the meshscatter object.

ufechner7 · June 25, 2024, 6:34pm

Maybe like this: KiteViewers.jl/src/common.jl at d97b75285136a9e77b3d354a84ca53c3234bb0cc · aenarete/KiteViewers.jl · GitHub

I updated KiteViewers.jl to the latest version of Makie.jl and heavily move objects around.

Ok, the type definitions are also important:

    points::Vector{Point{3, Float32}}
    positions::Observable{Vector{GeometryBasics.Point{3, Float32}}}

I first calculate the vector of the positions, and then assign them to a vector of observables.

    # move, scale and turn the cylinder correctly
    kv.positions[]   = calc_positions(kv.set.segments)
    kv.markersizes[] = calc_markersizes(kv.set.segments)
    kv.rotations[]   = calc_rotations(kv.set.segments)

sijo · June 25, 2024, 6:48pm

Are you looking for Observable? Cf Observables | Makie

The general idea is that you wrap your data vector in an Observable, then when you update the vector the plot updates automatically.

tamasgal · June 25, 2024, 7:02pm

Yes I did not realise that I can sneak them into a MeshScatter, I thought that I am supposed to interact with the object itself, that’s what I am doing so far (with color and markersize, which I access directly via fields and do not define them as Observables)

jules · June 25, 2024, 7:36pm

These attributes are always observables. If you pass in non-observables, they are converted. If you pass observables, they are used directly, so previous and later connections to them will have an effect on the plot object. So often it’s more convenient to pass in an already existing observable that you plan to mutate.