Web plotting when datapoints >> plot pixels

Makie is pretty great for plotting big datasets, in particular the datashader functionality datashader · Makie

But in the case of WGLMakie, you would first have to send the dataset to the client, which could be gigabytes.

Over in the Python world the Datashader + Holoviews + Plotly combination has a different approach: Datashader produces an image which is rendered in Plotly, and Holoviews hooks up Plotly zoom events to rerender the image. Example: https://twitter.com/pepijndevos/status/1509948476586663937?s=19 Docs: Interactivity — Datashader v0.14.4

This way you’re only sending kilobytes of data to the client rather than gigabytes, but you can still zoom in on the data and see increased details.

I think it would definitely be possible and useful to have access to this approach for big data notebooks, but I haven’t quite figured out all the plumbing, so I thought I’d share what I got and maybe someone has better ideas.

Basically I use Makie to generate a static plot, and use show to turn it into a base64 encoded data URL. I then set this as an image on Plotly, and hook up the plotly_relayout event to send the plot bounds to Julia which I use to set the limits on the Makie plot.

But I haven’t actually managed to close the loop where Plotly updates with the new Makie plot.

Here is the notebook I’m playing around with: JuliaHub

But in the case of WGLMakie, you would first have to send the dataset to the client, which could be gigabytes.

That’s not true, you’ll only send the final image!

1 Like

Then… what’s WebGL about it if it actually doesn’t render anything in the client?

I was actually trying to confirm how it works but if I open the web inspector after it loads it shows nothing and if I reload the page it stops working altogether…

I’d imagine for small datasets you would want it the other way. Where pixels >> datapoints you’d want to actually render on the client. You’d also very much want this behavior if you’re generating a static HTML page where you want to be able to zoom into plotted data. So now I have the opposite question…

Can you open an issue with an MWE for when it stops working?

I’d imagine for small datasets you would want it the other way

I think you should wait until at least we actually publish this feature.
I’m planning to write a blogpost explaining all of this.

1 Like

ok thanks