# Dealing with large numbers of points

**URL:** <https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826>\
**Category:** Visualization\
**Created:** [November 21, 2018, 5:15pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826 "2018-11-21T17:15:54Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 5:15pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/1 "2018-11-21T17:15:54Z")

</div>

I’m trying to use Plots.jl with GR backend to generate a scatter plot with a large number of points. However, 10,000 points fails to generate a plot in the Plots window and Juno generally crashes soon after.

Am I doing something wrong, or is the library unable to handle large numbers of points?

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [November 21, 2018, 5:40pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/2 "2018-11-21T17:40:20Z")

</div>

This was an issue in some earlier version. What’s your versions of Plots, GR and Julia?

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 6:05pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/3 "2018-11-21T18:05:08Z")

</div>

Julia 1.0.1  
Not sure how to find out about package versions, though I ran Pkg.add again on them and Plots seemed to have no change, while GR’s update reported “+ GR v0.34.1”

Still blank and crashed despite that, when I generated a second plot and tried to use Juno’s ability to navigate between plots.

For reference, I’m running:`x=rand(10000); y=rand(10000); scatter(x,y)`

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [November 21, 2018, 6:52pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/4 "2018-11-21T18:52:53Z")

</div>

I opened a similar thread some months ago:

> [@GR vs GLVisualize comparison?](https://discourse.julialang.org/t/gr-vs-glvisualize-comparison/3576/21):
>
> @Juan: and InspectDR doesn’t work either: […] What versions are you using? It worked for me with Julia v0.6.0-rc2.0 + Windows 7, and the following: julia\> Pkg.installed("InspectDR") v"0.2.0" julia\> Pkg.installed("Plots") v"0.12.0" Also: Have you recently done an update: julia\> Pkg.update() ?

My experience then was that GR was quite slow and didn’t deal well with large datasets.  
You can try with GLVisualize (now Makie) and InspectDR instead.

Now I’m not using Julia but still keeping and eye on it.  
What package is currently the best solution to deal with a large number of points? Say 10K, 100K o 1M points.  
Maybe with OnlineStats?

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [November 21, 2018, 7:03pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/5 "2018-11-21T19:03:24Z")

</div>

The GR backend should be completely capable of handling very large numbers of data points (have you seen this issue @jheinen?). The InspectDR backend is also focused on doing exactly that. 10,000 scatter points is nothing, though.  
Makie is not a backend to Plots. It is very good at handling large numbers of data though ([https://hackernoon.com/drawing-2-7-billion-points-in-10s-ecc8c85ca8fa](https://hackernoon.com/drawing-2-7-billion-points-in-10s-ecc8c85ca8fa) ).

@BioTurboNick could you try in the REPL to see if this is a juno interaction issue?

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [November 21, 2018, 7:09pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/6 "2018-11-21T19:09:09Z")

</div>

Nice, though that solution doesn’t just rely on Makie but on several custom (and complex for the average user) scripts to map the data from disk to memory.

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [November 21, 2018, 7:22pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/7 "2018-11-21T19:22:51Z")

</div>

Yes absolutely, it was not a suggested solution but a demonstration of Makie’s potential (it’s still fairly new)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 7:27pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/8 "2018-11-21T19:27:01Z")

</div>

I was using the Juno REPL. And… yeah, it works when run outside of Juno. Where should I report this problem?

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 7:29pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/9 "2018-11-21T19:29:15Z")

</div>

Is there a way to redirect plots output so it doesn’t use Juno’s window?

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [November 21, 2018, 7:41pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/10 "2018-11-21T19:41:18Z")

</div>

Yes you can turn the plot pane off in the settings. But first, try calling `gr(fmt = :png)` and see if it helps.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 7:53pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/11 "2018-11-21T19:53:19Z")

</div>

Found this advice: [Output · Plots](http://docs.juliaplots.org/latest/output/) but the environment variable doesn’t seem to work.

Turning the plot plane off in the Juno settings prevents it from using the plot plane, but no window is spawned to show results.

`gr(fmt = :png)` does not help

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [November 21, 2018, 8:03pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/12 "2018-11-21T20:03:37Z")

</div>

Oh I think the environment variable might be deprecated.  
It’s a bug, so try to report it on Plots.jl as an issue and ping pfitzseb (the Juno maintainer). Thanks. (BTW you get package versions with `]st`)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2018, 8:05pm UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/13 "2018-11-21T20:05:33Z")

</div>

Ah, it’s already an issue there, #1727: [Issue with GR plot in Atom · Issue #1727 · JuliaPlots/Plots.jl · GitHub](https://github.com/JuliaPlots/Plots.jl/issues/1727)

---

<div class="post-metadata">

**Author:** ![jheinen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jheinen/32/2787_2.png) [@jheinen](https://discourse.julialang.org/u/jheinen)\
**Post date:** [November 22, 2018, 9:16am UTC](https://discourse.julialang.org/t/dealing-with-large-numbers-of-points/17826/14 "2018-11-22T09:16:35Z")

</div>

[GR.jl v0.36.0-4-gfc0d1e6](https://github.com/jheinen/GR.jl/commit/fc0d1e67ef852a4d2e8d9007d96022ddddb09038) now takes respect of the plot pane setting in Atom.