InteractiveViz - an interactive visualization toolkit for large datasets

Package Announcements
1

Announcing InteractiveViz, an interactive visualization toolkit for working with large (even infinite) datasets.

Julia already has a rich set of plotting tools in the form of the Plots and Makie ecosystems, and various backends for these. InteractiveViz is not a replacement for Plots or Makie , but rather a graphics pipeline system developed on top of Makie with specific use cases in mind:

  • Interactively explore scatter plots or timeseries plots with tens of millions of data points.
  • Provide perceptually accurate summaries at large scale, allowing drill down to individual data points.
  • Allow generation of data points on demand through a graphics pipeline (e.g. infinite zoom into a fractal generated on demand), requiring computation only at a level of detail appropriate for display at the viewing resolution. Additional data points can be generated on demand when zooming or panning.

This package was partly inspired by the excellent Datashader package available in the Python ecosystem.

This package does not aim to provide comprehensive production quality plotting. It is aimed at interactive exploration of large datasets.

The package was developed primarily due to my own research use cases, and generalized to make it useful for the community. Feedback and contributions most welcome.

31 Likes
2

Great!

mandelbrot works on my Mac :+1:

3

I missed something like this for a long time! Will there be a way to hook into the machinery with custom types as the source for the plot?

Thanks for your work!

4

Yes, you can provide your own data source – simply a function that generates the data on demand. See demo source for an example. If you have specific API ideas on how you’d like your data types to be hooked in, I’d be glad to discuss those.

1 Like
5

Wow very nice!

Btw. GR plots does have a datashader for scatter plots - it even has part of it’s code cross-compiled to run in the browser, so they are interactive in HTML displays. Unfortunately, this is currently only available via GR.jl directly - the design of Plots.jl makes it hard to support it for the GR backend (though it may be possible, in principle).

CC @jheinen

6

Just tried the awesome Mandelbrot demo, and was wondering: Since the rendering is done on my GPU already, is there a way to tell InteractiveViz to use a CuArray to make the Mandelbrot computation run on my GPU as well? I think zooming it would be pretty much real-time, then. :slight_smile:

1 Like
7

Hmmm… interesting idea. It should be possible. I, unfortunately, don’t have a CUDA compatible GPU on my machine to test this with, but when I get a chance, I’ll prototype this to try…

Alternatively, I’m open to a PR which does this too :stuck_out_tongue:

8

Hm, maybe I’ll give it a try! :slight_smile:

Speaking of Mandelbrot sets - I just used InteractiveViz to render a Julia set, and was wondering - do we have an “official” Julia set for Julia yet?

1 Like
9

Oh, how did it not occur to me that the Julia set was the perfect one to include as a demo! :smiley:

Must add it in!

2 Likes
10

Wow, this looks really cool! You should look out for the “text in pixelspace” PR to AbstractPlotting, coming soon. Also, have you seen MakieLayout? It implements layouting and is pretty flexible with what you can lay out, and it might be interesting to integrate that with InteractiveViz - especially so that you can use InteractiveViz as part of a larger plotting pipeline.

Please let us know if you have any feature requests for Makie, or if you run into any bugs!

11

Thanks for the feedback and suggestions. Will certainly take a look at both, and coordinate more closely with the Makie development as the pipeline API matures.

12

Thank you for this package and I assume it is very helpful. But is it (not yet) possible to use it in the Juno environment? I got this message:

(@v1.4) pkg> add InteractiveViz
Updating registry at C:\Users\guent\.juliapro\JuliaPro_v1.4.1-1\registries\JuliaPro
ERROR: The following package names could not be resolved:

  • InteractiveViz (not found in project, manifest or registry)

I suppose I’m a little too impatient… :wink:

13

Try calling ]up beforehand, you might not have the latest registry?

14

The JuliaPro registry lags behind General, so InteractiveViz won’t have propagated there yet. You could add by URL:

]add https://github.com/org-arl/InteractiveViz.jl
15

I suppose so! And thanks for the advice! I had used the direct link in the past for other packages, but unfortunately I “forgot” this way! Oh dear, what is wrong with my memory…!? :relaxed: :+1:

Update:
I have updated my Julia-Juno-version and then started the download via the link. And now I get some error messages:

(@v1.4) pkg> add https://github.com/org-arl/InteractiveViz.jl
   Updating git-repo `https://github.com/org-arl/InteractiveViz.jl`
  Resolving package versions...
ERROR: Unsatisfiable requirements detected for package GLM [38e38edf]:
 GLM [38e38edf] log:
 ├─possible versions are: [1.0.0-1.0.2, 1.1.0-1.1.1, 1.2.0, 1.3.0-1.3.9] or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions [1.0.0-1.0.2, 1.1.0-1.1.1, 1.2.0, 1.3.0-1.3.9]
 ├─restricted by compatibility requirements with StatsMakie [65254759] to versions: 1.3.0-1.3.9
 │ └─StatsMakie [65254759] log:
 │   ├─possible versions are: [0.0.1-0.0.7, 0.1.0-0.1.5, 0.2.0-0.2.2] or uninstalled
 │   └─restricted by compatibility requirements with Makie [ee78f7c6] to versions: [0.1.4-0.1.5, 0.2.0-0.2.2]
 │     └─Makie [ee78f7c6] log:

I think I’ll have to wait a little longer until the package is made available regularly via JuliaPro. Thanks again for your advice!

16

I am loving this project. I am working with very large 2D datasets that I need to map and this is the only way to visualize them. Any way to add a colorbar, a title and to save the plot to a figure?

17

Yes. Makie has advanced quite a bit since this package was originally developed. I intend to use Makie for layout, and also add color-bar, title, etc, but haven’t found time recently to do it. Sometime soon…

4 Likes
18

Is there a way to add the colorbar and other things from Makie? Like together.
It is just that your package is the only one that seems to handle my data hahah

19

Also, it there a way to make the window a bit larger and to safe the plot?
Sorry for all the questions :slight_smile:

20

There should eventually be a nicer way to do this when I rework everything to work with MakieLayout, but this might work as a hack for now.

To resize current window to (2000, 1000):

let scene = ifigure(hold=true).scene
    resize!(scene, (2000, 1000))
    display(scene)
end

This works to add a colorbar:

Colorbar(ifigure(hold=true).scene; height=500, width=100, valign=:bottom, halign=:left)

but I haven’t figured out how to place it where you want yet.

2 Likes