# Seamless integration of Bumper.jl

**URL:** <https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916>\
**Category:** Performance\
**Created:** [June 20, 2024, 11:53am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916 "2024-06-20T11:53:49Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 20, 2024, 11:53am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/1 "2024-06-20T11:53:49Z")

</div>

Hi All,

I have a rather naive question. I have a function computing heuristic value of a state in planning using graph neural network. This function takes state, converts it to a graph uses network and return a single scalar value. So it seems like an ideal candidate for `Bumper.jl`.

So ideally, I would like to do something like `@no_escape(my code goes here)` and magically all allocation would happen through the bumper.

Is it possible? Is there a plan for this? I can see how this can be useful for example for training neural networks.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [June 20, 2024, 12:56pm UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/2 "2024-06-20T12:56:52Z")

</div>

Probably difficult, because the packages you use need to have explicit Bumper support, which most packages currently don’t have.

---

<div class="post-metadata">

**Author:** ![AMJ](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amj/32/214096_2.png) [@AMJ](https://discourse.julialang.org/u/AMJ)\
**Post date:** [June 20, 2024, 2:30pm UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/3 "2024-06-20T14:30:33Z")

</div>

You can use AllocArrays.jl.

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 21, 2024, 8:42am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/4 "2024-06-21T08:42:29Z")

</div>

That would make sense. I have tried to naively use AllocArrays, but no luck

```julia
julia> @benchmark map(states) do s
       with_allocator(buffer) do
               r = model(pddle(s))
               AllocArrays.reset!(buffer)
               r
           end
           end
BenchmarkTools.Trial: 6 samples with 1 evaluation.
 Range (min … max): 190.468 ms … 201.215 ms ┊ GC (min … max): 13.92% … 17.44%
 Time (median): 193.559 ms ┊ GC (median): 14.00%
 Time (mean ± σ): 194.199 ms ± 3.948 ms ┊ GC (mean ± σ): 14.67% ± 1.60%

  █ █ █ ██ █
  █▁▁▁█▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  190 ms Histogram: frequency by time 201 ms <

 Memory estimate: 985.04 MiB, allocs estimate: 20556.

julia> @benchmark map(model ∘ pddle, states)
BenchmarkTools.Trial: 6 samples with 1 evaluation.
 Range (min … max): 182.934 ms … 192.560 ms ┊ GC (min … max): 10.02% … 14.59%
 Time (median): 188.578 ms ┊ GC (median): 11.65%
 Time (mean ± σ): 188.544 ms ± 3.385 ms ┊ GC (mean ± σ): 11.97% ± 1.61%

  █ █ █ █ █ █
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁█▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁█ ▁
  183 ms Histogram: frequency by time 193 ms <

 Memory estimate: 985.04 MiB, allocs estimate: 20137.

```

---

<div class="post-metadata">

**Author:** ![AMJ](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amj/32/214096_2.png) [@AMJ](https://discourse.julialang.org/u/AMJ)\
**Post date:** [June 21, 2024, 8:57am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/5 "2024-06-21T08:57:06Z")

</div>

I think we should probably ask @ericphanson for more help. But in general you are still at the mercy of Bumper.jl limitations. For a full on control we should wait until mmtk (custom GC heuristics)is integrated as a gc backend then we can see how developers are able to use it.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 21, 2024, 9:05am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/6 "2024-06-21T09:05:29Z")

</div>

It’s hard to tell, are you actually using an AllocArray there?

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 21, 2024, 9:14am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/7 "2024-06-21T09:14:57Z")

</div>

How do I test this?  
It seems to me that I do not use it.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 21, 2024, 9:31am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/8 "2024-06-21T09:31:49Z")

</div>

Well it’s hard to say without your actual code. The way AllocArrays work is that you create an `AllocArray` yourself, pass it into your code, and then whenever the code calls `similar`, that call will use the bump allocator to produce a new AllocArray with bump-allocated memory. For example,

```julia
using Flux, AllocArrays
model = Chain(Dense(1 => 23, tanh), Dense(23 => 1; bias=true), only)
data = [[x] for x in -2:0.001f0:2]
alloc_data = AllocArray.(data)
function run_model(b, model, data)
    with_allocator(b) do
        result = sum(model, data)
        reset!(b)
        return result
    end
end

b = BumperAllocator(2^25) # 32 MiB
run_model(b, model, data)

```

The idea here is we can skip the intermediate allocations which may be hidden in library code, as long as that library code uses `similar`.

See [here](https://github.com/ericphanson/AllocArrays.jl/blob/9939bb8fae71f5e34abdd0f1a7beaded8c9c504a/test/flux.jl#L29-L147) for a more complicated example, using this for low-allocation inference of a Flux model.

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 21, 2024, 9:33am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/9 "2024-06-21T09:33:57Z")

</div>

Now I see, thanks a lot, this makes perfect sense.

That might be a bit complicated to change though.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 21, 2024, 9:35am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/10 "2024-06-21T09:35:31Z")

</div>

would it just be this?

> [@Tomas\_Pevny](#):
>
> ```julia
> map(states) do s
> with_allocator(buffer) do
> r = model(pddle(AllocArray(s)))
> AllocArrays.reset!(buffer)
> r
> end
> end
> 
> ```

edit: though make sure `r` is a scalar here, not a 1-element array or such! You can use a `CheckedAllocArray` for a slow version that verifies all memory accesses are safe (useful for testing).

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 21, 2024, 10:01am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/11 "2024-06-21T10:01:18Z")

</div>

Nice, this is neat.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 21, 2024, 10:35am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/12 "2024-06-21T10:35:28Z")

</div>

BTW if you get it working, I’d be interested to see the before-and-after benchmarking 🙂 . When I tried it on that Flux model I saw 100x less allocation, but similar runtime; I’m curious how it fares in other settings.

---

<div class="post-metadata">

**Author:** ![svretina](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/svretina/32/206157_2.png) [@svretina](https://discourse.julialang.org/u/svretina)\
**Post date:** [June 21, 2024, 11:08am UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/13 "2024-06-21T11:08:29Z")

</div>

sorry for the slight offtopic question,  
is `Bumber.jl` still developed? it seems that  
the last activity is 7 months ago

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 21, 2024, 12:01pm UTC](https://discourse.julialang.org/t/seamless-integration-of-bumper-jl/115916/14 "2024-06-21T12:01:05Z")

</div>

it seems pretty featureful at this point and none of the open issues look like bad bugs, so my assumption is that it is reasonably mature and code changes would be motivated by compat updates, new ideas, or bugfixes. Therefore 7 months of inactivity doesn’t seem like a bad sign. But @Mason could probably say more.