# Recording mouse position coordinates in plots?

Is there a way to click on a plot and have the coordinates at the pointer recorded (read)? (E.g. like, in R, the locator() function).
I mostly use Plots.jl but I may use a different package (and I would prefer interactivity, as with e.g. InspectDR or Pyplot).
Indeed, Plots + inspectdr() allows to show coordinates at mouse location (and even compare between different locations), but not to record them.

2 Likes

With PyPlot is easy, this is an example:

``````using DifferentialEquations
using PyPlot
using PyCall
@pyimport matplotlib as mpl

function f(dx, x, p, t)
dx[1] = x[2]
dx[2] = -sin(x[1])
end

fig, ax = plt.subplots()

title("Phase portrait in Julia.")
xlabel(L"\$x_1\$")
ylabel(L"\$x_2\$")
xlim([-2, 8])
ylim([-4, 4])

function on_button_press(event)
n = 300 #number of timepoints
t_sim = range(0, stop=1, length=n)
t_span = (0.0, 300.0)
x, y = event.xdata, event.ydata
u0 = [x, y]
prob = ODEProblem(f, u0, t_span)
sol = solve(prob)
plt.plot(sol(t_sim, idxs=1), sol(t_sim, idxs=2), "k-", markersize=10) # path
plt.plot(sol(t_sim, idxs=1)[1], sol(t_sim, idxs=2)[2], "*", markersize=10) # start
plt.plot(sol(t_sim, idxs=1)[end-1], sol(t_sim, idxs=2)[end], "o", markersize=10) # end
fig.canvas.draw()
end

fig.canvas.mpl_connect("button_press_event", on_button_press)
plt.show()

``````
1 Like

You mean something like this:
http://juliaplots.org/MakieReferenceImages/gallery//mouse_picking/index.html
?

2 Likes

### InspectDR

At the moment, InspectDR is not structured for this sort of thing - howeverâ€¦

### Broken Solution

(Though you might be able to figure it out)

In theory, you could â€śconnectâ€ť your own signal callback function as done in function `PlotWidget` of `src/gtk_top.jl`:

``````using InspectDR
using Gtk
PlotWidget = InspectDR.PlotWidget

#Create some plot:
x = collect(-10:10)
y = x.^2

#Display plot/construct GtkPlot object:
gplot = display(InspectDR.GtkDisplay(), mplot)

#Get a refrence to the base PlotWidget object:
pwidget = gplot.subplots[1]

@guarded function cb_mousepress_custom(w::Ptr{Gtk.GObject}, event::Gtk.GdkEventButton, pwidget::PlotWidget)
#Warning: .pos MIGHT be "nothing"!
@show pwidget.mouseover.pos
nothing #Ensure returns nothing
end

signal_connect(cb_mousepress_custom, pwidget.widget, "button-press-event", Nothing, (Ref{Gtk.GdkEventButton},), false, pwidget)
``````

But for some reason, I cannot get 2 signals to connect to the same widget.

### Alternative Hack

For the moment, you could â€śhack inâ€ť your own â€śevent handlerâ€ť in `src/gtk_input.jl`:

``````function handleevent_mousepress(::ISNormal, pwidget::PlotWidget, event::Gtk.GdkEventButton)
#	@show event.state, event.button, event.event_type
focus_strip(pwidget, event.x, event.y)
set_focus(pwidget) #In case not in focus

if 3==event.button
boxzoom_setstart(pwidget, event.x, event.y) #Changes state
elseif 1==event.button
if modifiers_pressed(event.state, MODIFIER_SHIFT)
mousepan_setstart(pwidget, event.x, event.y) #Changes state
elseif !modifiers_pressed(event.state, MODIFIERS_SUPPORTED) #Un-modified

handleevent_mousepress(pwidget, CtrlElement, event.x, event.y)
end
end
end
``````

ADD_CODE_HERE runs when the user uses the left mouse button WITHOUT a modifier key being pressed (NOT holding SHIFT, CTRL, or ALT).

Normally, InspectDR uses this event to move â€ścontrol elementsâ€ť of markers on the plot.

You can access the last known mouse position using: `pwidget.mouseover.pos`. This value is already re-mapped from `event` (screen coordinates). Warning: `pos` MIGHT be `nothing`!.

For the future, if you have questions relating to InspectDR, feel free to tag me directly (by typing in my handle: @MA_Laforge). I am more likely to notice your post this way.

Thanks @elsuizo for the suggestions. I tried a minimal working example with PyPlot. My goal is not only to mark the plot by mouse clicking, but to also save the coordinates of the click location.
I donâ€™t know pyplot enough, following the example I have got this workaround, which prints the coordinates on the REPL:

``````function on_button_press(event)
x, y = event.xdata, event.ydata
u0 = [x, y]
println(u0)
plt.plot(x, y, "+", markersize=10)
fig.canvas.draw()
end

x = 1:12;
y = rand(12);
fig, ax = plt.subplots()
plt.plot(x,y)
fig.canvas.mpl_connect("button_press_event", on_button_press)
plt.show()
``````

With `prinln(u0)` I get the values printed in the REPL output; after finishing I copy and edit them. Is there a way to get them directly in a vector or matrix or dataframe? I tried with `return(u0)`, but thatâ€™s not working.

1 Like

Hi @MA_Laforge, thanks for the code. However, the first example seems not to produce anything. I get the plot, but nothing happens when clicking on the graph. I am probably missing something.

As for the second solution I get an error for ::ISNormal:
`ERROR: UndefVarError: ISNormal not defined`
What is it meant for?

Does the `ginput()` function in PyPlot do what you want?

1 Like

Thanks @sdanisch, thatâ€™s quite near what I need. I have never used Makie so far and Iâ€™ll search the docs how to display my data.
But first a question, is it possible to interactively zoom and pan the graph?

Yes, zooming works in Makie!

Yes, that does the trick! And it also works with `Plots`, using the `pyplot()` backend.
So now I have two possible easy to implement solutions, this is one, the other Makie.

Correct @davide:

### First example

The first example does not work at the moment. I think it is supposed to work, but the `"button-press-event"` signal already has a connection in the InspectDR code. I think this is a bug with Gtk. I am almost certain you are supposed to be able to connect multiple callback functions to the same â€śsignalâ€ť. I merely included the example in case you better understood how Gtk worked, and knew how to get around the problem.

### Second example

As for the second example: that one will definitively work. I call it a â€śhackâ€ť because you actually have to add your code (or possibly a call to your user-defined function) directly in InspectDRâ€™s `src/gtk_input.jl` file.

### Sightly less hack-y

â€¦But since we are talking Julia here, I suppose you could use the â€śdynamic patchingâ€ť (guerrilla patch/monkey patch) technique to overwrite the function from your own code, instead of directly modifying the InspectDR code:

``````using InspectDR
using Gtk

function InspectDR.handleevent_mousepress(::InspectDR.ISNormal, pwidget::InspectDR.PlotWidget, event::Gtk.GdkEventButton)
#	@show event.state, event.button, event.event_type
InspectDR.focus_strip(pwidget, event.x, event.y)
InspectDR.set_focus(pwidget) #In case not in focus

if 3==event.button
InspectDR.boxzoom_setstart(pwidget, event.x, event.y) #Changes state
elseif 1==event.button
if InspectDR.modifiers_pressed(event.state, InspectDR.MODIFIER_SHIFT)
InspectDR.mousepan_setstart(pwidget, event.x, event.y) #Changes state
elseif !InspectDR.modifiers_pressed(event.state, InspectDR.MODIFIERS_SUPPORTED) #Un-modified

@show pwidget.mouseover.pos #NOTE: Typically a Point2D struct, but might be nothing.

InspectDR.handleevent_mousepress(pwidget, InspectDR.CtrlElement, event.x, event.y)
end
end
end
``````

Note that I now had to add a direct reference to `InspectDR` for all function calls/structure definitions/etc defined in the InspectDR module. These functions/structures/â€¦ are not exported by InspectDR - so they are not otherwise available from your own functionâ€™s scope.

Hope that helps.

### FYI: Regarding `ISNormal`

I am not certain you actually wanted to know the backstory of `ISNormal`, but since you askedâ€¦

An object of type `ISNormal` is a subtype of the `InputState` type.

I use these structures to handle mouse and keyboard events in InspectDR by modelling a form of state machine. Depending on what the user has done in the past, the plot widget might be in different states.

For example, I have the following states:

• `ISNormal`: Default state for a widget. Not doing anything special.
• `ISMovingMarker`: In the process of moving a â€ścontrol pointâ€ť that controls the position of a â€śposition markerâ€ť.
• `ISMovingÎ”Info`: In the process of moving an â€śinfo boxâ€ť displaying the difference data between two â€śposition markersâ€ť.
• `ISPanningData`: The user â€śshift-clickedâ€ť on the plot widget, so now mouse moves should be used to pan the data window.
• `ISSelectingArea`: The user â€śright-clickedâ€ť on the plot widget, so now mouse moves should be used define the zoom-in area.

I donâ€™t really think you need to know this to get things to work at the moment, but you might find the implementation interesting if ever you need to do more complicated work with mouse/keyboard events.

I am wondering how to use `ginput()` under `pyplot()` backend. When I run it, it generates the following error:

``````julia> using Plots; pyplot(); plot(randn(100))
julia> ginput()
ERROR: UndefVarError: ginput not defined
Stacktrace:
[1] top-level scope
@ REPL[17]:1
``````

Thanks!

Hi @BVPs

what I do is:

``````julia> using Plots
julia> using PyPlot, PyCall
julia> pyplot() # Plots backend
julia> Plots.plot(randn(100))
julia> ginput()
``````

Just checked (it was some time I didnâ€™t use it), it also works with:

``````julia> using Plots
julia> using PyPlot: ginput
julia> pyplot() # Plots backend
julia> plot(randn(100))
julia> ginput()
``````

Thanks a lot, @davide ! BTW, is there any similar functionality under `gr()` and `plotlyjs()` backends?

By the way, `ginput()` as @davide suggested works if I run it from a terminal, but if I run it from vscode, it does not work. It generates the following warning and does not return any picked coordinate:

``````sys:1: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
``````