Recording mouse position coordinates in plots?

Specific Domains Visualization
, ,
1

Is there a way to click on a plot and have the coordinates at the pointer recorded (read)? (E.g. like, in R, the locator() function).
I mostly use Plots.jl but I may use a different package (and I would prefer interactivity, as with e.g. InspectDR or Pyplot).
Indeed, Plots + inspectdr() allows to show coordinates at mouse location (and even compare between different locations), but not to record them.

2 Likes
2

With PyPlot is easy, this is an example:

using DifferentialEquations
using PyPlot
using PyCall
@pyimport matplotlib as mpl

function f(dx, x, p, t)
   dx[1] = x[2]
   dx[2] = -sin(x[1])
end

fig, ax = plt.subplots()

title("Phase portrait in Julia.")
xlabel(L"$x_1$")
ylabel(L"$x_2$")
xlim([-2, 8])
ylim([-4, 4])

function on_button_press(event)
   n = 300 #number of timepoints
   t_sim = range(0, stop=1, length=n)
   t_span = (0.0, 300.0)
   x, y = event.xdata, event.ydata
   u0 = [x, y]
   prob = ODEProblem(f, u0, t_span)
   sol = solve(prob)
   plt.plot(sol(t_sim, idxs=1), sol(t_sim, idxs=2), "k-", markersize=10) # path
   plt.plot(sol(t_sim, idxs=1)[1], sol(t_sim, idxs=2)[2], "*", markersize=10) # start
   plt.plot(sol(t_sim, idxs=1)[end-1], sol(t_sim, idxs=2)[end], "o", markersize=10) # end
   fig.canvas.draw()
end

fig.canvas.mpl_connect("button_press_event", on_button_press)
plt.show()
1 Like
3

You mean something like this:
http://juliaplots.org/MakieReferenceImages/gallery//mouse_picking/index.html
?

3 Likes
4

InspectDR

At the moment, InspectDR is not structured for this sort of thing - however…

Broken Solution

(Though you might be able to figure it out)

In theory, you could “connect” your own signal callback function as done in function PlotWidget of src/gtk_top.jl:

using InspectDR
using Gtk
PlotWidget = InspectDR.PlotWidget

#Create some plot:
x = collect(-10:10)
y = x.^2

mplot = InspectDR.Multiplot(title="Quadratic")
plot = add(mplot, InspectDR.Plot2D)
	add(plot, x, y)

#Display plot/construct GtkPlot object:
gplot = display(InspectDR.GtkDisplay(), mplot)

#Get a refrence to the base PlotWidget object:
pwidget = gplot.subplots[1]

#Define your own callback function:
@guarded function cb_mousepress_custom(w::Ptr{Gtk.GObject}, event::Gtk.GdkEventButton, pwidget::PlotWidget)
	#Warning: .pos MIGHT be "nothing"!
	@show pwidget.mouseover.pos
	nothing #Ensure returns nothing
end

#Add another "button-press-event" callback function:
signal_connect(cb_mousepress_custom, pwidget.widget, "button-press-event", Nothing, (Ref{Gtk.GdkEventButton},), false, pwidget)

But for some reason, I cannot get 2 signals to connect to the same widget.

Alternative Hack

For the moment, you could “hack in” your own “event handler” in src/gtk_input.jl:

function handleevent_mousepress(::ISNormal, pwidget::PlotWidget, event::Gtk.GdkEventButton)
#	@show event.state, event.button, event.event_type
	focus_strip(pwidget, event.x, event.y)
	set_focus(pwidget) #In case not in focus

	if 3==event.button
		boxzoom_setstart(pwidget, event.x, event.y) #Changes state
	elseif 1==event.button
		if modifiers_pressed(event.state, MODIFIER_SHIFT)
			mousepan_setstart(pwidget, event.x, event.y) #Changes state
		elseif !modifiers_pressed(event.state, MODIFIERS_SUPPORTED) #Un-modified

			#!!! ADD_CODE_HERE !!!

			handleevent_mousepress(pwidget, CtrlElement, event.x, event.y)
		end
	end
end

ADD_CODE_HERE runs when the user uses the left mouse button WITHOUT a modifier key being pressed (NOT holding SHIFT, CTRL, or ALT).

Normally, InspectDR uses this event to move “control elements” of markers on the plot.

You can access the last known mouse position using: pwidget.mouseover.pos. This value is already re-mapped from event (screen coordinates). Warning: pos MIGHT be nothing!.

5

@davide:

For the future, if you have questions relating to InspectDR, feel free to tag me directly (by typing in my handle: @MA_Laforge). I am more likely to notice your post this way.

6

Thanks @elsuizo for the suggestions. I tried a minimal working example with PyPlot. My goal is not only to mark the plot by mouse clicking, but to also save the coordinates of the click location.
I don’t know pyplot enough, following the example I have got this workaround, which prints the coordinates on the REPL:

function on_button_press(event)
   x, y = event.xdata, event.ydata
   u0 = [x, y]
   println(u0)
   plt.plot(x, y, "+", markersize=10)
   fig.canvas.draw()
end

x = 1:12;
y = rand(12);
fig, ax = plt.subplots()
plt.plot(x,y)
fig.canvas.mpl_connect("button_press_event", on_button_press)
plt.show()

With prinln(u0) I get the values printed in the REPL output; after finishing I copy and edit them. Is there a way to get them directly in a vector or matrix or dataframe? I tried with return(u0), but that’s not working.

1 Like
7

Hi @MA_Laforge, thanks for the code. However, the first example seems not to produce anything. I get the plot, but nothing happens when clicking on the graph. I am probably missing something.

As for the second solution I get an error for ::ISNormal:
ERROR: UndefVarError: ISNormal not defined
What is it meant for?

8

Does the ginput() function in PyPlot do what you want?

1 Like
9

Thanks @sdanisch, that’s quite near what I need. I have never used Makie so far and I’ll search the docs how to display my data.
But first a question, is it possible to interactively zoom and pan the graph?

10

Yes, zooming works in Makie!

11

Yes, that does the trick! And it also works with Plots, using the pyplot() backend.
So now I have two possible easy to implement solutions, this is one, the other Makie.

12

Correct @davide:

First example

The first example does not work at the moment. I think it is supposed to work, but the "button-press-event" signal already has a connection in the InspectDR code. I think this is a bug with Gtk. I am almost certain you are supposed to be able to connect multiple callback functions to the same “signal”. I merely included the example in case you better understood how Gtk worked, and knew how to get around the problem.

Second example

As for the second example: that one will definitively work. I call it a “hack” because you actually have to add your code (or possibly a call to your user-defined function) directly in InspectDR’s src/gtk_input.jl file.

Sightly less hack-y

…But since we are talking Julia here, I suppose you could use the “dynamic patching” (guerrilla patch/monkey patch) technique to overwrite the function from your own code, instead of directly modifying the InspectDR code:

using InspectDR
using Gtk

function InspectDR.handleevent_mousepress(::InspectDR.ISNormal, pwidget::InspectDR.PlotWidget, event::Gtk.GdkEventButton)
#	@show event.state, event.button, event.event_type
	InspectDR.focus_strip(pwidget, event.x, event.y)
	InspectDR.set_focus(pwidget) #In case not in focus

	if 3==event.button
		InspectDR.boxzoom_setstart(pwidget, event.x, event.y) #Changes state
	elseif 1==event.button
		if InspectDR.modifiers_pressed(event.state, InspectDR.MODIFIER_SHIFT)
			InspectDR.mousepan_setstart(pwidget, event.x, event.y) #Changes state
		elseif !InspectDR.modifiers_pressed(event.state, InspectDR.MODIFIERS_SUPPORTED) #Un-modified

		#!!! ADD CODE HERE !!!
@show pwidget.mouseover.pos #NOTE: Typically a Point2D struct, but might be nothing.


			InspectDR.handleevent_mousepress(pwidget, InspectDR.CtrlElement, event.x, event.y)
		end
	end
end

Note that I now had to add a direct reference to InspectDR for all function calls/structure definitions/etc defined in the InspectDR module. These functions/structures/… are not exported by InspectDR - so they are not otherwise available from your own function’s scope.

Hope that helps.

13

FYI: Regarding ISNormal

I am not certain you actually wanted to know the backstory of ISNormal, but since you asked…

An object of type ISNormal is a subtype of the InputState type.

I use these structures to handle mouse and keyboard events in InspectDR by modelling a form of state machine. Depending on what the user has done in the past, the plot widget might be in different states.

For example, I have the following states:

  • ISNormal: Default state for a widget. Not doing anything special.
  • ISMovingMarker: In the process of moving a “control point” that controls the position of a “position marker”.
  • ISMovingΔInfo: In the process of moving an “info box” displaying the difference data between two “position markers”.
  • ISPanningData: The user “shift-clicked” on the plot widget, so now mouse moves should be used to pan the data window.
  • ISSelectingArea: The user “right-clicked” on the plot widget, so now mouse moves should be used define the zoom-in area.

I don’t really think you need to know this to get things to work at the moment, but you might find the implementation interesting if ever you need to do more complicated work with mouse/keyboard events.

14

I am wondering how to use ginput() under pyplot() backend. When I run it, it generates the following error:

julia> using Plots; pyplot(); plot(randn(100))
julia> ginput()
ERROR: UndefVarError: ginput not defined
Stacktrace:
 [1] top-level scope
   @ REPL[17]:1

Thanks!

15

Hi @BVPs

what I do is:

julia> using Plots
julia> using PyPlot, PyCall
julia> pyplot() # Plots backend
julia> Plots.plot(randn(100))
julia> ginput()

Just checked (it was some time I didn’t use it), it also works with:

julia> using Plots
julia> using PyPlot: ginput
julia> pyplot() # Plots backend
julia> plot(randn(100))
julia> ginput()
16

Thanks a lot, @davide ! BTW, is there any similar functionality under gr() and plotlyjs() backends?

1 Like
17

By the way, ginput() as @davide suggested works if I run it from a terminal, but if I run it from vscode, it does not work. It generates the following warning and does not return any picked coordinate:

sys:1: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.