# How to release the extra buffer after insert! an array?

**URL:** <https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385>\
**Category:** General Usage\
**Tags:** question\
**Created:** [June 29, 2024, 6:10am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385 "2024-06-29T06:10:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Blumenkranz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blumenkranz/32/210335_2.png) [@Blumenkranz](https://discourse.julialang.org/u/Blumenkranz)\
**Post date:** [June 29, 2024, 6:10am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/1 "2024-06-29T06:10:23Z")

</div>

I found that julia allocates a lot when `insert!` an array for the first time. But for the second or subsequent `insert!` operations, the memory allocation becomes 0. I guess that the initial `insert!` pre-allocates memory to enhance array modification speed (am I correct?). This allocated memory is often quite substantial, which can be burdensome for large number of arrays. Considering that the modification frequency of each array is not high, is there a way to release this pre-allocated memory?

---

<div class="post-metadata">

**Author:** ![Tarny\_GG\_Channie](https://avatars.discourse-cdn.com/v4/letter/t/3bc359/32.png) [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Post date:** [June 29, 2024, 6:20am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/2 "2024-06-29T06:20:02Z")

</div>

sizehint! should be fine. If you want extra control, maybe a custom array type would be better.

---

<div class="post-metadata">

**Author:** ![Blumenkranz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blumenkranz/32/210335_2.png) [@Blumenkranz](https://discourse.julialang.org/u/Blumenkranz)\
**Post date:** [June 29, 2024, 6:38am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/4 "2024-06-29T06:38:29Z")

</div>

There seems to be a minimum amount of pre-allocated memory in Julia, as `sizehint!` will pre-allocate at least this amount of memory, even if the amount I request is lower than this value. See:

```julia
a = collect(1:1000)
@allocated sizehint!(a,1001)

```

returns `17280`. An interesting fact is that `insert!` also allocates the same:

```julia
b = collect(1:1000)
@allocated insert!(b,2,1)

```

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [June 29, 2024, 6:46am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/5 "2024-06-29T06:46:56Z")

</div>

1. try `sizehint!` on an _empty_ array
2. try the upcoming release (v1.11), as possibly some things changed

---

<div class="post-metadata">

**Author:** ![Blumenkranz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blumenkranz/32/210335_2.png) [@Blumenkranz](https://discourse.julialang.org/u/Blumenkranz)\
**Post date:** [June 29, 2024, 7:02am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/6 "2024-06-29T07:02:28Z")

</div>

I hope to preserve the dynamic nature of arrays, so `sizehint!` an empty array does not meet the requirements. Perhaps there will be other methods? Anyway, looking forward to v1.11!

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 29, 2024, 8:44am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/7 "2024-06-29T08:44:38Z")

</div>

> [@Blumenkranz](#):
>
> I guess that the initial `insert!` pre-allocates memory to enhance array modification speed (am I correct?).

Yes that’s correct. It’s a common technique for amortizing the cost of resizing a dynamic array, since resizing must eventually copy the data in the array around. IIRC the default is to double the capacity (=reserved memory) whenever the array needs to grow its backing memory.

> [@Blumenkranz](#):
>
> Considering that the modification frequency of each array is not high, is there a way to release this pre-allocated memory?

If you allocate an empty array, IIRC it reserves a capacity of 32 elements. Calling `sizehint!` on that is about as efficient as you can get, short of allocating the array with the correct capacity in the first place and calling `empty!` afterwards. The reserved capacity should be retained in that case.

> [@Blumenkranz](#):
>
> I hope to preserve the dynamic nature of arrays, so `sizehint!` an empty array does not meet the requirement

I’m not sure what you mean here - why not? `sizehint!` is all about the reserved capacity, not the exposed size. You can have an array with a capacity of let’s say 5000 elements, and only expose 500 of those.

---

<div class="post-metadata">

**Author:** ![Blumenkranz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blumenkranz/32/210335_2.png) [@Blumenkranz](https://discourse.julialang.org/u/Blumenkranz)\
**Post date:** [June 29, 2024, 9:16am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/8 "2024-06-29T09:16:48Z")

</div>

Thanks for your explaination. When I mentioned that `sizehint!` doesn’t meet my needs, it’s because there are a large number of arrays in the program, and to avoid out of memory errors, I want the pre-allocated memory to be released immediately after resizing the arrays. I think this could be achieved by copying the elements from the original array to a newly created array with precisely pre-allocated size, but this approach may lead to low efficiency (maybe that’s the best I can get?).

---

<div class="post-metadata">

**Author:** ![Tarny\_GG\_Channie](https://avatars.discourse-cdn.com/v4/letter/t/3bc359/32.png) [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Post date:** [June 29, 2024, 9:36am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/9 "2024-06-29T09:36:30Z")

</div>

It depends on your use case. If you know the size of array in advance, you can use static array.  
However, if you want the most compact possible array, then it’s difficult. Even C malloc is allowed to allocate more memory than you actually requested. Many structs are also padded.

Perhaps using vcat can help? But honestly, there is no guarantee. Julia array is allowed to be larger than its determined size.

---

<div class="post-metadata">

**Author:** ![Blumenkranz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blumenkranz/32/210335_2.png) [@Blumenkranz](https://discourse.julialang.org/u/Blumenkranz)\
**Post date:** [June 29, 2024, 9:41am UTC](https://discourse.julialang.org/t/how-to-release-the-extra-buffer-after-insert-an-array/116385/10 "2024-06-29T09:41:29Z")

</div>

I see… Thanks a lot!