# Gadfly scatter plot crashing

**URL:** <https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066>\
**Category:** New to Julia\
**Tags:** plotting, gadfly\
**Created:** [July 25, 2023, 9:55am UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066 "2023-07-25T09:55:32Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 9:55am UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/1 "2023-07-25T09:55:33Z")

</div>

Hello, I have a data frame with 2 columns, each has 2 millions points. When I try to plot it crashes Jupyter. I am surprised because R and Python can handle it.

Here is my command

> df2=CSV.read(“/home/alessandro/Data/MAexperiment/24-07-2023\_Deepvariant\_standing\_variation\_analysis\_based\_on\_HiFi/RefCall\_all\_GQ\_VAF”, DataFrame; header = false)

> rename!(df2,[:RefCall,:VAF,:GQ])  
> t=Gadfly.plot(df2, x=“GQ”, y=“VAF”, Geom.point())

Is there something I can do? I feel like it should be able to handle it no? Or am I delusional?

Thanks

EDIT: I can plot the data using

> using Plots  
> gr()  
> @df df2 Plots.scatter(:GQ, :VAF)

but this causes massive performance issues. I suspect therefore there is somethign wrong with Jupyter/Brave browser

EDIT 2: with Python it takes 576 ms to plot

EDIT 3: this works but it’s very slow, in a .jl script, takes around 1 minute. I am aware of time to first plot, but there is no improvement over time, it still takes seconds to plot.

> using DataFrames
> 
> using CSV
> 
> df4=CSV.read(“/home/alessandro/Data/MAexperiment/24-07-2023\_Deepvariant\_standing\_variation\_analysis\_based\_on\_HiFi/RefCall\_all\_GQ\_VAF”, DataFrame; header = false)
> 
> rename!(df4,[:RefCall,:VAF,:GQ])
> 
> using Gadfly
> 
> t=Gadfly.plot(df4, x=“GQ”, y=“VAF”,Geom.point, Theme(background\_color=color(“white”),grid\_color=color(“white”)))
> 
> using Cairo
> 
> using Fontconfig
> 
> t|\> PNG(“density.png”,30cm,25cm)

EDIT 4: it also crashes Pluto… So far only VisualStudio is able to render my plot, though very slowly. I don’t believe it’s my computer fault, here is my hardware:  
product: Intel(R) Xeon(R) CPU E3-1505M v5 @ 2.80GHz  
64GiB System Memory

My issue is reminiscent of this one: [Why is Julia's graphics system so slow? - #32 by evan-wehi](https://discourse.julialang.org/t/why-is-julias-graphics-system-so-slow/68750/32)

I am not good enough in computing to understand all the implications. Can you tell me if what’s happening is actually normal behaviour and I shouldn’t use Julia for data exploration? I am really puzzled.

Thanks and sorry for the many edits, I kept doing research.

---

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [July 25, 2023, 2:29pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/2 "2023-07-25T14:29:19Z")

</div>

(disclaimer: not a plotting expert)

I suspect Gadfly might not be the right plotting package if you need to plot millions of points. Gadfly produces vector images, and in the case of a scatter plot the number of elements to render is linear in the number of points you’re plotting. Contrast with a raster based plotting package, where the plot image size doesn’t depend on the number of points in your scatter plot. Of course the image buffer has to be updated for each point you’re plotting, but the image size itself, the number of pixels to be drawn, etc., don’t grow with more points.

I think Julia is a _great_ tool for data analysis! But perhaps try out a non-vector plotting package if you need to plot millions of points in a scatter plot. (Side note: why do you need to plot millions of points in a single scatter plot? That’s a very crowded plot, the points will be drastically overlapping each other and you won’t see the density of points anyway unless you make each point nearly transparent, in which case you’re basically just generating a 2d density plot anyway?)

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 2:39pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/3 "2023-07-25T14:39:14Z")

</div>

Thanks, you are correct that using a 2d histogram is much more interesting! (and it works much better, it’s rendered almost instantaneously). I am switching to Plots, seem much faster indeed. If you have any other recommendation, don’t hesitate.

---

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [July 25, 2023, 2:41pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/4 "2023-07-25T14:41:40Z")

</div>

Glad that’s working for you!

Also just for curiosity I tried plotting 1 million points with Python+Pandas and found that it took only ~100ms for the plot call to return in the repl—but when I tried to view and manipulate the interactive plot pane, it froze and I got an OverflowError! I think plotting millions of points is just hard 🙂

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 2:44pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/5 "2023-07-25T14:44:50Z")

</div>

Indeed  
I work in genomics and we have millions, or even billions data points to deal with, in case you would wonder where that many points come from. They are just data from genomes, and genomes are, well huge. Or in fact very small … it depends on how you see it, since your whole genome can be stored on a simple usb stick.

I have never made up my mind around this “are our genomes small or big things?” 😃

---

<div class="post-metadata">

**Author:** ![bjarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bjarthur/32/9638_2.png) [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Post date:** [July 25, 2023, 3:19pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/6 "2023-07-25T15:19:43Z")

</div>

did you try a 2D histogram in Gadfly? that should be fast too. i’d also suggest trying out Makie too.

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 3:23pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/7 "2023-07-25T15:23:32Z")

</div>

yes, the 2d histogram is much faster. i will check out Makie

---

<div class="post-metadata">

**Author:** ![Rudi79](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rudi79/32/3884_2.png) [@Rudi79](https://discourse.julialang.org/u/Rudi79)\
**Post date:** [July 25, 2023, 3:38pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/8 "2023-07-25T15:38:26Z")

</div>

Maybe try out:

> **[GitHub - org-arl/InteractiveViz.jl: Interactive visualization tools for Julia](https://github.com/org-arl/InteractiveViz.jl)**
>
> Interactive visualization tools for Julia. Contribute to org-arl/InteractiveViz.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 5:13pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/9 "2023-07-25T17:13:18Z")

</div>

Oh that seems VERY interesting

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 8:40pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/10 "2023-07-25T20:40:05Z")

</div>

hem, how to display the plot, when in the REPL? (I gave up on Jupyter I find the integration very clunky and I spend more time arranging the cells and writing pretty paragraphs in Markdowns than actually analyzing the data.

For example, minimal example of Makie

> f = Figure(backgroundcolor = :tomato)  
> Scene (800px, 600px):  
> 0 Plots  
> 0 Child Scenes

display(f) returns the same

---

<div class="post-metadata">

**Author:** ![bjarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bjarthur/32/9638_2.png) [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Post date:** [July 25, 2023, 9:06pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/11 "2023-07-25T21:06:21Z")

</div>

which Makie backend are you using?

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 25, 2023, 9:12pm UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/12 "2023-07-25T21:12:52Z")

</div>

GLMakie

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 26, 2023, 2:21am UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/13 "2023-07-26T02:21:44Z")

</div>

GLMakie

---

<div class="post-metadata">

**Author:** ![Axze-rgb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/axze-rgb/32/50986_2.png) [@Axze-rgb](https://discourse.julialang.org/u/Axze-rgb)\
**Post date:** [July 28, 2023, 6:46am UTC](https://discourse.julialang.org/t/gadfly-scatter-plot-crashing/102066/14 "2023-07-28T06:46:27Z")

</div>

Hello  
Okay it seems my conda and jupyter were broken, works like a charm now (no idea how / what I did broke)