# Performance optimization of a custom binning function

**URL:** <https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616>\
**Category:** Performance\
**Created:** [December 13, 2022, 6:50pm UTC](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616 "2022-12-13T18:50:35Z")\
**Posts on this page:** 4\
**Page:** 3

<div class="post-metadata">

**Author:** ![alex-s-gardner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex-s-gardner/32/30210_2.png) [@alex-s-gardner](https://discourse.julialang.org/u/alex-s-gardner)\
**Post date:** [December 15, 2022, 6:53pm UTC](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616/42 "2022-12-15T18:53:42Z")

</div>

Here’s the final version that I settled on, thanks again for everyone’s inputs:

```julia
"""
    bin(x::Vector{<:Number}, y::Array{<:Number}, xbin_edges::Union{Vector{<:Number},StepRangeLen{}}; method::F = mean) where {F<:Function}

Fast binning of `y` as a function of `x`. Returns `x_binned`, `y_binned`, `bin_count`. `method` 
specifies approached used for aggregating binned values. NOTE: `x_binned` == 0 and 
`y_binned` == 0 when `bin_count` == 0.

# see https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-funciton/91616
"""
function bin(
    x::Vector{<:Number}, 
    y::Array{<:Number}, 
    xbin_edges::Union{Vector{<:Number},StepRangeLen{}};
    method::F = mean,
    ) where {F<:Function}

    # find bin breakpoints
    p = sortperm(vcat(x, xbin_edges))
    bins = findall(>(length(x)), p)

    # initialize outputs
    bin_count = Int.(diff(bins).-1)
    x_binned = zeros(eltype(x), length(bin_count))
    y_binned = zeros(eltype(y), length(bin_count), size(y, 2))

    # calculate binned metrics
    for i = findall(bin_count .> 0)
        x_binned[i] = method(@view(x[p[bins[i]+1:bins[i+1]-1]]))
        y_binned[i] = method(@view(y[p[bins[i]+1:bins[i+1]-1]]))
    end
    
    return x_binned, y_binned, bin_count 
end

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 15, 2022, 7:20pm UTC](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616/43 "2022-12-15T19:20:30Z")

</div>

> [@alex-s-gardner](#):
>
> `x = rand(100); y = rand(100); xbin_edges = 0:0.1:1`

You forgot the `@view()` around the slices, which here give a some additional speedup and memory savings:

```julia
julia> @btime bin($x,$y,$xbin_edges);
  3.500 μs (56 allocations: 8.72 KiB)

```

after

```julia
               x_binned[i] = method(@view(x[p[bins[i]+1:bins[i+1]-1]]))
               y_binned[i] = method(@view(y[p[bins[i]+1:bins[i+1]-1]]))

```

I get:

```julia
julia> @btime bin($x,$y,$xbin_edges);
  2.786 μs (36 allocations: 5.97 KiB)

```

---

<div class="post-metadata">

**Author:** ![alex-s-gardner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex-s-gardner/32/30210_2.png) [@alex-s-gardner](https://discourse.julialang.org/u/alex-s-gardner)\
**Post date:** [December 15, 2022, 7:26pm UTC](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616/45 "2022-12-15T19:26:08Z")

</div>

Good catch… previous post edited

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [December 15, 2022, 11:08pm UTC](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616/46 "2022-12-15T23:08:36Z")

</div>

I found that puzzling as well and went on to read the backstory: [Make `findall` faster for AbstractArrays by nalimilan · Pull Request #37177 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/37177). Food for thought.

[Previous page](https://discourse.julialang.org/t/performance-optimization-of-a-custom-binning-function/91616.md?page=2)