Makie.jl Mouse Interaction Question/Discussion

I recently got Julia 1.0.0 up and running with my favorite packages and tried out Makie and interactive plotting just worked!

I have a question and potential suggestion about the mouse interaction model and curious to see other peoples’ views.

I noticed that the default mouse interactions are

  • pan = right-click + drag
  • XY-zoom = scroll wheel

The default interaction behavior of pyqtgraph is something special in the world of visualization tools. Its mouse interactions are

  • pan = left-click + drag
  • X-zoom = right-click + drag left/right
  • Y-zoom = right-click + drag up/down

I found that model to be extremely useful in ‘navigating’ long time-series of multi-channel data.

I’m wondering…

  1. How difficult would it be to achieve this behavior with Makie.jl? Can I just call a few functions and get this behavior?

  2. Are others interested in this kind of interaction model and have others used pyqtgraph or its ‘zooming’ model?

3 Likes

I have no details, but i’d assume that current interaction is hard coded. As the code is largely uncommented, you should follow event processing in the backends.

1 Like

you should follow event processing in the backends.

What do you mean by that?

Well the good news is, that it’s not hard coded.

So you can just do e.g.:

scene = Scene()
cam = cam2D!(scene) # you can also get the cam from an existing scene
cam[:panbutton] = Mouse.left

Changing zoom is a bit more involved, since it’s a whole different way of zooming.
But the camera is composable, so you can take the camera code here:


and change it to something like:

function mycam2d!(scene::SceneLike; kw_args...)
    cam_attributes, rest = merged_get!(:cam2d, scene, Attributes(kw_args)) do
        Theme(
            area = Signal(FRect(0, 0, 1, 1), name = "area"),
            zoomspeed = 0.10f0,
            zoombutton = nothing,
            panbutton = Mouse.right,
            selectionbutton = (Keyboard.space, Mouse.left),
            padding = 0.001
        )
    end
    cam = from_dict(Camera2D, cam_attributes)
    # remove previously connected camera
    disconnect!(camera(scene))
    my_zoom!(scene, cam) # add your own zoom
    add_pan!(scene, cam)
    correct_ratio!(scene, cam)
    selection_rect!(scene, cam, cam_attributes[:selectionbutton])
    cameracontrols!(scene, cam)
    cam
end

I hope the code in the file will help to write my_zoom

1 Like

I’m not sure how one would do this, but another common (and IMO useful) zoom for 2D plots is the “box zoom”: the user draws a rectangular box around the region of interest (by left clicking and dragging) and that region expands to take the whole space. Double-clicking brings it back to the original state: see Plotly for an example implementation. Not sure how easy/hard this is to implement and whether it should be a opt-in or default behavior (I tend to think this kind of things should be easy to access / default, as only advanced users will write code to customize mouse interaction).

That’s already implemented:

selectionbutton = (Keyboard.space, Mouse.left)

and I do hope I wrote it in a way that it’s easily customizable… After all the camera is basically also just a recipe, which can be swapped out with your own code building up on the existing camera building blocks!

Cool! Just two remarks:

  • I think the “box zoom” code still needs some updating, I’m on master of AbstractPlotting and I get:
MethodError: no method matching ntuple(::getfield(AbstractPlotting, Symbol("##191#192")){Vec{2,Float32}}, ::Type{Val{2}})
Closest candidates are:
  ntuple(::F, ::Integer) where F at tuple.jl:133
  ntuple(::Any, ::Val{0}) at tuple.jl:155
  ntuple(::Any, ::Val{1}) at tuple.jl:156
  ...
absrect(::GeometryTypes.HyperRectangle{2,Float32}) at /home/pietro/.julia/packages/AbstractPlotting/ZHerD/src/camera/camera2d.jl:143 	31 	2.8k 	3d
Julia for live control of USB3.0 camera and DAC/ADC boards, with GUI? 1 	Usage 	11 	208 	5d
Too many ways to do the same thing in Julia 1 	Usage 	16 	643 	9d
Is it possible to override keyword arguments for specific types? 2
question
	Usage 	7 	158 	10d
There are 95 unread and 14 new topics remaining, or browse other topics in Visualization
sdanisch

Cool! Just two remarks:

    I think the “box zoom” code still needs some updating, I’m on master of AbstractPlotting and I get:

MethodError: no method matching ntuple(::getfield(AbstractPlotting, Symbol("##191#192")){Vec{2,Float32}}, ::Type{Val{2}})
Closest candidates are:
  ntuple(::F, ::Integer) where F at tuple.jl:133
  ntuple(::Any, ::Val{0}) at tuple.jl:155
  ntuple(::Any, ::Val{1}) at tuple.jl:156
  ...
absrect(::GeometryTypes.HyperRectangle{2,Float32}) at /home/pietro/.julia/packages/AbstractPlotting/ZHerD/src/camera/camera2d.jl:143

  • I think it’s important to have this easily customizable, esp. in the case of touch pad users. On many laptops there is a mechanism to block touch pad when typing so Space + Mouse Left is effectively disabled
  • I think it’s important to have this easily customizable, esp. in the case of touch pad users. On many laptops there is a mechanism to block touch pad when typing so Space + Mouse Left is effectively disabled

That’s why you can just change it :stuck_out_tongue: It’s even themeable, so you can change it globally!

1 Like

This is really cool, I’m slowly wrapping my head around it. Here is the PR to fix the behavior on Julia 1.0.

I tried scatter(rand(100), rand(100), selectionbutton=Mouse.left) but it doesn’t work, what’s the correct way to do this?

ah, I don’t pass through the attributes:


But otherwise, this should be how it works:

scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left))

I should remove the NT and just use NamedTuples on 1.0!

Mhm, still doesn’t work:

julia> p = scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left))

julia> p.theme[:cam2d].attributes
Dict{Symbol,Reactive.Signal} with 6 entries:
  :padding         => 5422: "padding" = 0.001 Any 
  :selectionbutton => 5421: "selectionbutton" = (space::Button = 32, left::Button = 0) Any 
  :zoomspeed       => 5418: "zoomspeed" = 0.1 Any 
  :panbutton       => 5420: "panbutton" = right Any 
  :area            => 5417: "area" = GeometryTypes.HyperRectangle{2,Float32}(Float32[0.0, 0.0], Float32[1.0, 1.0]) Any 
  :zoombutton      => 5419: "zoombutton" = nothing Any 

But if I change it manually it works:

p.theme[:cam2d][:selectionbutton] = Mouse.left

But I think as soon as this is figured out, it definitely deserves a section in the docs, it’s really nice to be able to define custom interactive behavior at the single plot level or globally with a theme.

Oh, and a last thing: did you also implement a “reset button” to go back to the original plot size?

Sorry for the many questions but I’m getting really excited about the possibilities Makie offers.

1 Like

Me:

ah, I don’t pass through the attributes:

What I meant is, that scatter(rand(100), rand(100), cam2d = NT(selectionbutton=Mouse.left)) currently doesn’t work!

"reset button” to go back to the original plot size?

Ah no, that is missing but should be easy to implement!

3 Likes