Makie.jl Mouse Interaction Question/Discussion

standarddeviant · September 1, 2018, 5:19am

I recently got Julia 1.0.0 up and running with my favorite packages and tried out Makie and interactive plotting just worked!

I have a question and potential suggestion about the mouse interaction model and curious to see other peoples’ views.

I noticed that the default mouse interactions are

pan = right-click + drag
XY-zoom = scroll wheel

The default interaction behavior of pyqtgraph is something special in the world of visualization tools. Its mouse interactions are

pan = left-click + drag
X-zoom = right-click + drag left/right
Y-zoom = right-click + drag up/down

I found that model to be extremely useful in ‘navigating’ long time-series of multi-channel data.

I’m wondering…

How difficult would it be to achieve this behavior with Makie.jl? Can I just call a few functions and get this behavior?
Are others interested in this kind of interaction model and have others used pyqtgraph or its ‘zooming’ model?

lobingera · September 1, 2018, 12:28pm

I have no details, but i’d assume that current interaction is hard coded. As the code is largely uncommented, you should follow event processing in the backends.

sdanisch · September 3, 2018, 7:26am

you should follow event processing in the backends.

What do you mean by that?

Well the good news is, that it’s not hard coded.

github.com

JuliaPlots/AbstractPlotting.jl/blob/master/src/camera/camera2d.jl#L15


      
              panbutton::Node{Union{ButtonTypes, Vector{ButtonTypes}}}
              padding::Node{Float32}
              last_area::Node{Vec{2, Int}}
              update_limits::Node{Bool}
          end
          
          """
              cam2d!(scene::SceneLike, kwargs...)
          
          Creates a 2D camera for the given Scene.
          """
          function cam2d!(scene::SceneLike; kw_args...)
              cam_attributes = merged_get!(:cam2d, scene, Attributes(kw_args)) do
                  Attributes(
                      area = Node(FRect(0, 0, 1, 1)),
                      zoomspeed = 0.10f0,
                      zoombutton = nothing,
                      panbutton = Mouse.right,
                      selectionbutton = (Keyboard.space, Mouse.left),
                      padding = 0.001,
                      last_area = Vec(size(scene)),

So you can just do e.g.:

scene = Scene()
cam = cam2D!(scene) # you can also get the cam from an existing scene
cam[:panbutton] = Mouse.left

Changing zoom is a bit more involved, since it’s a whole different way of zooming.
But the camera is composable, so you can take the camera code here:

github.com

JuliaPlots/AbstractPlotting.jl/blob/master/src/camera/camera2d.jl#L19


      
          end
          
          """
              cam2d!(scene::SceneLike, kwargs...)
          
          Creates a 2D camera for the given Scene.
          """
          function cam2d!(scene::SceneLike; kw_args...)
              cam_attributes = merged_get!(:cam2d, scene, Attributes(kw_args)) do
                  Attributes(
                      area = Node(FRect(0, 0, 1, 1)),
                      zoomspeed = 0.10f0,
                      zoombutton = nothing,
                      panbutton = Mouse.right,
                      selectionbutton = (Keyboard.space, Mouse.left),
                      padding = 0.001,
                      last_area = Vec(size(scene)),
                      update_limits = false,
                  )
              end
              cam = from_dict(Camera2D, cam_attributes)

and change it to something like:

function mycam2d!(scene::SceneLike; kw_args...)
    cam_attributes, rest = merged_get!(:cam2d, scene, Attributes(kw_args)) do
        Theme(
            area = Signal(FRect(0, 0, 1, 1), name = "area"),
            zoomspeed = 0.10f0,
            zoombutton = nothing,
            panbutton = Mouse.right,
            selectionbutton = (Keyboard.space, Mouse.left),
            padding = 0.001
        )
    end
    cam = from_dict(Camera2D, cam_attributes)
    # remove previously connected camera
    disconnect!(camera(scene))
    my_zoom!(scene, cam) # add your own zoom
    add_pan!(scene, cam)
    correct_ratio!(scene, cam)
    selection_rect!(scene, cam, cam_attributes[:selectionbutton])
    cameracontrols!(scene, cam)
    cam
end

I hope the code in the file will help to write my_zoom

piever · September 3, 2018, 8:34am

I’m not sure how one would do this, but another common (and IMO useful) zoom for 2D plots is the “box zoom”: the user draws a rectangular box around the region of interest (by left clicking and dragging) and that region expands to take the whole space. Double-clicking brings it back to the original state: see Plotly for an example implementation. Not sure how easy/hard this is to implement and whether it should be a opt-in or default behavior (I tend to think this kind of things should be easy to access / default, as only advanced users will write code to customize mouse interaction).

sdanisch · September 3, 2018, 8:39am

That’s already implemented:

selectionbutton = (Keyboard.space, Mouse.left)

and I do hope I wrote it in a way that it’s easily customizable… After all the camera is basically also just a recipe, which can be swapped out with your own code building up on the existing camera building blocks!

piever · September 3, 2018, 8:53am

Cool! Just two remarks:

I think the “box zoom” code still needs some updating, I’m on master of AbstractPlotting and I get:

MethodError: no method matching ntuple(::getfield(AbstractPlotting, Symbol("##191#192")){Vec{2,Float32}}, ::Type{Val{2}})
Closest candidates are:
  ntuple(::F, ::Integer) where F at tuple.jl:133
  ntuple(::Any, ::Val{0}) at tuple.jl:155
  ntuple(::Any, ::Val{1}) at tuple.jl:156
  ...
absrect(::GeometryTypes.HyperRectangle{2,Float32}) at /home/pietro/.julia/packages/AbstractPlotting/ZHerD/src/camera/camera2d.jl:143 	31 	2.8k 	3d
Julia for live control of USB3.0 camera and DAC/ADC boards, with GUI? 1 	Usage 	11 	208 	5d
Too many ways to do the same thing in Julia 1 	Usage 	16 	643 	9d
Is it possible to override keyword arguments for specific types? 2
question
	Usage 	7 	158 	10d
There are 95 unread and 14 new topics remaining, or browse other topics in Visualization
sdanisch

Cool! Just two remarks:

    I think the “box zoom” code still needs some updating, I’m on master of AbstractPlotting and I get:

MethodError: no method matching ntuple(::getfield(AbstractPlotting, Symbol("##191#192")){Vec{2,Float32}}, ::Type{Val{2}})
Closest candidates are:
  ntuple(::F, ::Integer) where F at tuple.jl:133
  ntuple(::Any, ::Val{0}) at tuple.jl:155
  ntuple(::Any, ::Val{1}) at tuple.jl:156
  ...
absrect(::GeometryTypes.HyperRectangle{2,Float32}) at /home/pietro/.julia/packages/AbstractPlotting/ZHerD/src/camera/camera2d.jl:143

I think it’s important to have this easily customizable, esp. in the case of touch pad users. On many laptops there is a mechanism to block touch pad when typing so Space + Mouse Left is effectively disabled

sdanisch · September 3, 2018, 8:56am

I think it’s important to have this easily customizable, esp. in the case of touch pad users. On many laptops there is a mechanism to block touch pad when typing so Space + Mouse Left is effectively disabled

That’s why you can just change it It’s even themeable, so you can change it globally!

piever · September 3, 2018, 9:07am

This is really cool, I’m slowly wrapping my head around it. Here is the PR to fix the behavior on Julia 1.0.

I tried scatter(rand(100), rand(100), selectionbutton=Mouse.left) but it doesn’t work, what’s the correct way to do this?

sdanisch · September 3, 2018, 9:13am

ah, I don’t pass through the attributes:

https://github.com/JuliaPlots/AbstractPlotting.jl/blob/master/src/basic_recipes/basic_recipes.jl#L400
But otherwise, this should be how it works:

scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left))

I should remove the NT and just use NamedTuples on 1.0!

piever · September 3, 2018, 9:25am

Mhm, still doesn’t work:

julia> p = scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left))

julia> p.theme[:cam2d].attributes
Dict{Symbol,Reactive.Signal} with 6 entries:
  :padding         => 5422: "padding" = 0.001 Any 
  :selectionbutton => 5421: "selectionbutton" = (space::Button = 32, left::Button = 0) Any 
  :zoomspeed       => 5418: "zoomspeed" = 0.1 Any 
  :panbutton       => 5420: "panbutton" = right Any 
  :area            => 5417: "area" = GeometryTypes.HyperRectangle{2,Float32}(Float32[0.0, 0.0], Float32[1.0, 1.0]) Any 
  :zoombutton      => 5419: "zoombutton" = nothing Any

But if I change it manually it works:

p.theme[:cam2d][:selectionbutton] = Mouse.left

But I think as soon as this is figured out, it definitely deserves a section in the docs, it’s really nice to be able to define custom interactive behavior at the single plot level or globally with a theme.

piever · September 3, 2018, 9:29am

Oh, and a last thing: did you also implement a “reset button” to go back to the original plot size?

Sorry for the many questions but I’m getting really excited about the possibilities Makie offers.

sdanisch · September 3, 2018, 9:30am

Me:

ah, I don’t pass through the attributes:

What I meant is, that scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left)) currently doesn’t work!

"reset button” to go back to the original plot size?

Ah no, that is missing but should be easy to implement!

Topic		Replies	Views
Advance mouse interaction with Makie? General Usage package , makie	3	273	June 2, 2023
Makie interactions with nested plot Visualization makie , interactivity	0	386	April 26, 2022
Interactivity of Makie plots Tooling makie , interactivity	10	3412	July 5, 2024
[ANN] Mousetrap, the new Julia GUI Engine, now supports GLMakie & macOS Package Announcements package , announcement	9	2616	December 1, 2023
How to get the updated coordinates of vertices in the interaction with mouse using Makie.jl Specific Domains plotting , makie , glmakie	2	281	October 5, 2022

Makie.jl Mouse Interaction Question/Discussion

Related topics