As an offshoot of a thread about Makie histograms, I’ve tried to benchmark StatsBase fit(Histogtam) like so: julia> using StatsBase julia> @btime fit(Histogram, v, 0.0:0.1:1.0) setup=(Random.seed!(1234); v = rand(1_000_000)) 31.720 ms (2 allocations: 224 bytes) Histogram{Int64, 1, Tuple{StepRang…

Maybe check out this? [image] GitHub - Moelf/FHist.jl: A pure Julia 1D/2D histogram package that focus on... A pure Julia 1D/2D histogram package that focus on speed and thread-safe. - GitHub - Moelf/FHist.jl: A pure Julia 1D/2D histogram package that focus on speed and thread-safe. …

Yep, it’s even faster than the for loop version: julia> @btime Hist1D(v, 0.0:0.1:1.0) setup=(Random.seed!(1234); v = rand(1_000_000)) 6.645 ms (6 allocations: 544 bytes) edges: 0.0:0.1:1.0 bin counts: [99542, 99942, 99836, 100537, 99777, 99271, 99582, 100479, 100476, 100558] total count: 1000000 …

Slightly off-topic - I think the JuliaStats ecosystem has matured nicely, but also needs some new contributors to forge the way forward. The challenge has always been that existing owners are too busy to review everything and they do need to keep the ecosystem working. This would be exactly the kin…

gotta go fast: Inline and optimize `push!()` when one-shot histogramming by Moelf · Pull Request #87 · Moelf/FHist.jl · GitHub

I tried and failed: Pull requests · JuliaStats/StatsBase.jl · GitHub Issues · JuliaStats/StatsBase.jl · GitHub

[image] Dan: That’s really good. Maybe it should replace the StatsBase implementation. convert() should already work, so you can take my histogram and disard some information and go back to StatsBase.Histogram. My understanding is that hep use cases are very fringe in the grand scheme of thi…

[image] jling: hep use cases are very fringe in the grand scheme of things Don’t think it is so fringe. HEP practitioners might be more fringe, but the statistics/computation needs of every field seem to converge quite surprisingly. I agree that keeping the square weights is also important i…

That’s really a bummer, seeing all the efforts you have put in. I also work in a field where histograms are breakfast, lunch and dinner (astroparticle physics) and I do everything related with FHist.jl, it’s awesome. I wish JuliaStats/StatsBase would appreciate the need for a proper and fast implem…

[image] Dan: Maybe we should delineate the algorithms according to memory complexity: on-line algos (dynamic histogram) 1-pass algos 2-pass algos random access… And have a histogram version for each of these complexities. The first exists in: GitHub - joshday/OnlineStats.jl: ⚡ Single-pass …

[image] jling: In an ideal world I’d love to have the best histogram ecosystem in Julia Another niche important to be best in is the countmap area, also in StatsBase. Investing in StatsBase is important to draw/keep power users.

Speeding up `fit(Histogram)`

General Usage Performance

tbeason November 17, 2023, 2:16pm 2

Maybe check out this?

Topic		Replies	Views
Weighted histogram ~2x as slow in Julia vs. Python Performance question , statistics , python	13	1321	January 20, 2022
Use a Histogram as a function General Usage	4	2264	October 21, 2019
Why is hist not defined after loading StatsBase? New to Julia question	10	2760	May 13, 2018
Query on histc and histogram counts Statistics	5	3104	January 9, 2017
Hist() and hits() in Julia? Statistics question	2	4043	December 30, 2016

Speeding up `fit(Histogram)`

Related topics