Getting a 3D position from mouse click. Makie.jl

Hello, I am using Makie.jl to plot molecular shapes. Right now I am using it mostly as a passive tool: read a file with xyz coordinates and plot them. I would like to add some more interaction to it. So my main question is:

Upon right-click, I can get coordinates to create a new “Atom object”. However, those are 2D coordinates (which I assume are the ones projected onto the screen). Is there a way to back-project those to get an approximate position in the 3D space?

No, because there is an infinite ray of 3D coordinates under your mouse. You’d have to intersect it with something meaningful to you. Or do you mean finding the 3D position of some object that you clicked on? That should be possible. I’m not sure from your description which one you’re after.

Thanks. Yes, I understand that there is no unique way to perform that 2D → 3D projection. I would provide the “depth” by some criteria, such as proximity to other objects. For example, upon clicking I want a new object to appear under the mouse, constrained to be as close as possible to a nearby object.

I’ve done a bunch of this mouseposition to ray to intersection point business for DataInspector. All of the code for that is currently here: https://github.com/JuliaPlots/Makie.jl/blob/master/src/interaction/inspector.jl

view_ray(scene) will give you an (origin, ray_direction) in 3d space. There’s also closest_point_on_line which returns a point on a line (between A and B) and ray_triangle_intersection which returns a point on a triangle. I also have some code to find a point on the ray closest to another point which I haven’t pushed yet:

function relative_inverse_project(scene, pos)
    origin, dir = view_ray(scene)
    relative_inverse_project(pos, origin, dir)
end
function relative_inverse_project(pos, origin, dir)
    # This solves the following triangle:
    # pos ------ output
    #      \   |
    #       \  |
    #        \ | ↑ dir
    #         \|
    #        origin
    # where pos is the position of some anchor point and 
    # origin, dir define a ray (with dir assumed to be normalized)
    origin .+ sqrt(norm(pos .- origin) - dot(pos .- origin, dir)) * dir
end

Maybe some of those will be helpful.

2 Likes

Thank you! view_ray does exactly what I wanted! Now it’s just a matter of figuring out the best depth to place the objects!