Which low latency library for drawing polygon on image + extracting values?

I’m looking for a solution similar to R’s clickpoly from spatstat.geom (for those familiar with it). The goal is to let a user draw a polygon on a displayed image (greyscale or heatmap) by clicking the desired vertex points, to then extract the values of the image pixels within the polygon and do some statistics. It would be nice if the outline of the polygon is traced during drawing (so not only the vertex point are visible but also the edges as the user draws). The polygon can either close automatically after drawing a fixed number (e.g. 4) of corner points, or the user can click and drag a rectangle on the plot, or the user could click any desired number of corners and then close on right click - that is of minor importance at this point.

Any opinions on the best library to accomplish this (Makie, PlotlyJS, others…) ?

Getting the lowest latency is crucial here: the user should only do this once at the beginning of a large dataset processing job (after waiting shortly for some preprocessing to complete), so any lengthy TTFP would be very noticeable. I began implementing this last month in Makie, but I still noticed quite a significant latency (on 0.16.4, though I should probably check for updates, also in terms of documentation as I struggled to get a working implementation), which made me reconsider. For instance, this “Drawing shapes with a mouse on cartesian plots” in the Python Plotly reference (Shapes with Python (plotly.com)) looks like it would suit my use case, but I’m not sure this is possible from PlotlyJS.jl?

I would be happy to contribute to improved documentation/tutorials once I have worked this out.